Spatio-Temporal Branching for Motion Prediction using Motion Increments
Open accessAbstract
Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques, which often struggle to model the complex dynamics of human motion. Recent deep learning-based methods have achieved success by learning spatio-temporal representations of motion, but these models often overlook the reliability of motion data. Additionally, the temporal and spatial dependencies of skeleton nodes are distinct. The temporal relationship captures motion information over time, while the spatial relationship describes body structure and the relationships between different nodes. In this paper, we propose a novel spatio-temporal branching network using incremental information for HMP, which decouples the learning of temporal-domain and spatial-domain features, extracts more motion information, and achieves complementary cross-domain knowledge learning through knowledge distillation. Our approach effectively reduces noise interference and provides more expressive information for characterizing motion by separately extracting temporal and spatial features. We evaluate our approach on standard HMP benchmarks and outperform state-of-the-art methods in terms of prediction accuracy. Code is available at https://github.com/JasonWang959/STPMP.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
International Encyclopedia of Statistical Science
Lovric, M. (2011) · Springer eBooks · doi:10.1007/978-3-642-04898-2 · Cited by 1,696 · see citing works
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera
von Marcard, T., Henschel, R. D., Black, M. J., Rosenhahn, B., & Pons‐Moll, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_37 · Cited by 1,062 · see citing works
On Human Motion Prediction Using Recurrent Neural Networks
Martínez, J., Black, M. J., & Romero, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.497 · Cited by 1,019 · see citing works
Luo, W., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00376 · Cited by 644 · see citing works
Learning Trajectory Dependencies for Human Motion Prediction
Mao, W., Liu, M.-M., Salzmann, M., & Li, H. (2019) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2019.00958 · Cited by 591 · see citing works
Zhang, R., Zhu, F., Liu, J., & Liu, G. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2936913 · Cited by 427 · see citing works
Deep Representation Learning for Human Motion Prediction and Classification
Bütepage, J., Black, M. J., Kragić, D., & Kjellström, H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.173 · Cited by 414 · see citing works
Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction
Li, M., Chen, S.-H., Zhao, Y., Zhang, Y., Wang, Y.-F., & Tian, Q. (2020) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr42600.2020.00029 · Cited by 382 · see citing works
Predicting future locations with hidden Markov models
Mathew, W., Raposo, R., & Martins, B. (2012) · doi:10.1145/2370216.2370421 · Cited by 381 · see citing works
Convolutional Sequence to Sequence Model for Human Dynamics
Li, C., Zhang, Z., Lee, W. S., & Lee, G. H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00548 · Cited by 364 · see citing works
History Repeats Itself: Human Motion Prediction via Motion Attention
Mao, W., Liu, M., & Salzmann, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_28 · Cited by 322 · see citing works
Neural state machine for character-scene interactions
Starke, S., Zhang, H., Komura, T., & Saito, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356505 · Cited by 291 · see citing works
Chen, S., Liu, B., Feng, C., Vallespi-Gonzalez, C., & Wellington, C. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2984780 · Cited by 290 · see citing works
MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction
Dang, L., Nie, Y.-W., Long, C., Zhang, Q., & Li, G.-Q. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.01127 · Cited by 284 · see citing works
Adversarial Geometry-Aware Human Motion Prediction
Gui, L.-Y., Wang, Y.-X., Liang, X., & Moura, J. M. F. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_48 · Cited by 276 · see citing works
A Spatio-temporal Transformer for 3D Human Motion Prediction
Aksan, E., Kaufmann, M., Cao, P., & Hilliges, O. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00066 · Cited by 268 · see citing works
Space-Time-Separable Graph Convolutional Network for Pose Forecasting
Sofianos, T., Sampieri, A., Franco, L., & Galasso, F. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.01102 · Cited by 214 · see citing works
Human Motion Prediction via Spatio-Temporal Inpainting
Hernandez, A., Gall, J., & Moreno, F. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00723 · Cited by 196 · see citing works
Efficient Nonlinear Markov Models for Human Motion
Lehrmann, A., Gehler, P., & Nowozin, S. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.171 · Cited by 195 · see citing works
Human motion prediction for human-robot collaboration
Liu, H., & Wang, L. (2017) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2017.04.009 · Cited by 193 · see citing works
A Neural Temporal Model for Human Motion Prediction
Gopalakrishnan, A., Mali, A., Kifer, D., Giles, L., & Ororbia, A. G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01239 · Cited by 185 · cited 2× · see citing works
“RNNs [6, 9, 11, 13, 27] have been used to model the temporal dynamics of human motion, while CNNs [2, 7, 22] have been used to model the spatial information.”
MethodMa, T., Nie, Y.-W., Long, C., Zhang, Q., & Li, G. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00633 · Cited by 170 · see citing works
Learning Progressive Joint Propagation for Human Motion Prediction
Cai, Y., Huang, L., Wang, Y., Cham, T., Cai, J., Yuan, J., Liu, J., Yang, X., Zhu, Y., Shen, X., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_14 · Cited by 169 · see citing works
Context-Aware Human Motion Prediction
Corona, E., Pumarola, A., Alenyà, G., & Moreno-Noguer, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00702 · Cited by 163 · see citing works
Unhelkar, V. V., Lasota, P. A., Tyroller, Q., Buhai, R.-D., Marceau, L., Deml, B., & Shah, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2812906 · Cited by 151 · see citing works
Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction
Zhong, C.-Y., Hu, L., Zhang, Z., Ye, Y.-J., & Xia, S. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00634 · Cited by 131 · see citing works
Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning
Lee, S., Kim, H. G., Hwi Choi, D., Kim, H.-I., & Ro, Y. M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00307 · Cited by 129 · see citing works
BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
Kundu, J. N., Gor, M., & Babu, R. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018553 · Cited by 128 · see citing works
Ego-Pose Estimation and Forecasting As Real-Time PD Control
Yuan, Y., & Kitani, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01018 · Cited by 117 · see citing works
Anticipating human activities for reactive robotic response
Koppula, H. S., & Saxena, A. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696634 · Cited by 101 · see citing works
Efficient human motion prediction using temporal convolutional generative adversarial network
Cui, Q., Sun, H., Kong, Y., Zhang, X., & Li, Y. (2020) · Information Sciences · doi:10.1016/j.ins.2020.08.123 · Cited by 93 · see citing works
Skeleton Graph Scattering Networks for 3D Skeleton-based Human Motion Prediction
Li, M., Chen, S.-H., Liu, Z., Zhang, Z., Xie, L.-X., Tian, Q., & Zhang, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00101 · Cited by 35 · see citing works
Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptation
Qiang, W., Li, J., Zheng, C., Su, B., & Xiong, H. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3112815 · Cited by 25 · see citing works
Self-citationMeta Attention-Generation Network for Cross-Granularity Few-Shot Learning
Qiang, W., Li, J., Su, B., Fu, J., Xiong, H., & Wen, J.-R. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01760-7 · Cited by 19 · see citing works
Self-citationAuxiliary task guided mean and covariance alignment network for adversarial domain adaptation
Qiang, W., Li, J., Zheng, C., & Su, B. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107066 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 10
- Katerina Fragkiadaki, Sergey Levine, Panna Felsen, and Jitendra Malik. 2015. Recurrent network models for human dynamics. In Proceedings of the IEEE inter- national conference on computer vision. 4346-4354.
- Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In International conference on machine learning. PMLR, 1243-1252.
- Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. 2013. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence 36, 7 (2013), 1325-1339.
- Maosen Li, Siheng Chen, Zijing Zhang, Lingxi Xie, Qi Tian, and Ya Zhang. 2022. Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VI. Springer, 18-36.
- Vladimir Pavlovic, James M Rehg, and John MacCormick. 2000. Learning switch- ing linear models of human motion. Advances in neural information processing systems 13 (2000).
- Judea Pearl. 2001. Direct and indirect effects. In Proceedings of the seventeenth con- ference on uncertainty in artificial intelligence. Morgan Kaufmann, San Francisco, 411-420.
- Graham W Taylor, Geoffrey E Hinton, and Sam Roweis. 2006. Modeling human motion using binary latent variables. Advances in neural information processing systems 19 (2006).
- Herwin Van Welbergen, Ben JH Van Basten, Arjan Egges, Zs M Ruttkay, and Mark H Overmars. 2010. Real time animation of virtual humans: a trade-off between naturalness and control. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 2530-2554.
- Jack Wang, Aaron Hertzmann, and David J Fleet. 2005. Gaussian process dynam- ical models. Advances in neural information processing systems 18 (2005).
- Yujie Zhou, Haodong Duan, Anyi Rao, Bing Su, and Jiaqi Wang. 2023. Self- supervised Action Representation Learning from Partial Spatio-Temporal Skele- ton Sequences. arXiv preprint arXiv:2302.09018 (2023). · arXiv
Cited by
Sorted by citations
Chen, S., Peng, C., Yang, B., Lin, J., Zhou, L., Jiang, Z., Liu, Z., Liu, Y., & Tang, L. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00083-5 · Cited by 9 · see citing works
- Authors
- Wang, J., Zhou, Y., Qiang, W., Ba, Y., Su, B., & Wen, J.-R.
- Affiliations
- Renmin University of China (CN)
- Institute of Software (CN)
- Venue
- Proceedings of the 31st ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2023-10-26
- Pages
- 4290-4299
- OpenAlex
- W4385968244
- Cited by
- 11
- References
- 37
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 4
- FWCI
- 4.71
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61976206, 61832017)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- human motion prediction, knowledge distillation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
