Directed Acyclic Graph Neural Network for Human Motion Prediction
Abstract
Human motion prediction is essential in human-robot interaction. Current research mostly considers the joint dependencies but ignores the bone dependencies and their relationship in the human skeleton, thus limiting the prediction accuracy. To address this issue, we represent the human skeleton as a directed acyclic graph with joints as vertexes and bones as directed edges. Then, we propose a novel directed acyclic graph neural network (DA-GNN) that follows the encoder-decoder structure. The encoder is stacked by multiple encoder blocks, each of which includes a directed acyclic graph computational operator (DA-GCO) to update joint and bone attributes based on the relationship between joint and bone dependencies in the observed human states, and a temporal update operator (TUO) to update the temporal dynamics of joints and bones in the same observation. After progressively implementing the above update process, the encoder outputs the final update result, fed into the decoder. The decoder includes a directed acyclic graph-based gated recurrent unit (DAG-GRU) and a multi-layered perceptron (MLP) to predict future human states sequentially. To the best of our knowledge, this is the first time to introduce the relationship between bone and joint dependencies in human motion prediction. Our experimental evaluations on two datasets, CMU Mocap and Human 3.6m, prove that DA-GNN outperforms current models. Finally, we showcase the efficacy of DA-GNN in a realistic HRI scenario.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GNN models sensor topological relationships via graph structures, provides explicit spatial correlation inference, and is used for whole-body motion intent network analysis.
“| GNN | Model sensor topological relationships via graph | Explicit spatial correlation inference | Whole-body motion intent network analysis 270-272 |”
From Recent advances in intelligent wearable systems: from multiscale biomechanical features towards human motion intent prediction (2026) · cites this paper as [270]
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
Graph Neural Networks: A Review of Methods and Applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2018) · arXiv · doi:10.48550/arxiv.1812.08434 · Cited by 6,880 · see citing works
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · see citing works
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1× · see citing works
“Each category contains six motion sequences, where five ) [46]: is a public large-scale human motion dataset.”
BackgroundSocial LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.110 · Cited by 3,584 · see citing works
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., & Alahi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00240 · Cited by 2,466 · see citing works
(2016) · Springer Handbooks · doi:10.1007/978-3-319-32552-1 · Cited by 2,122 · see citing works
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133 · see citing works
Gaussian Process Dynamical Models for Human Motion
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1167 · Cited by 1,024 · see citing works
On Human Motion Prediction Using Recurrent Neural Networks
Martínez, J., Black, M. J., & Romero, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.497 · Cited by 1,019 · see citing works
Skeleton-Based Action Recognition With Directed Graph Neural Networks
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00810 · Cited by 979 · see citing works
Interaction Networks for Learning about Objects, Relations and Physics
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1612.00222 · Cited by 969 · see citing works
The hierarchical hidden Markov model: Analysis and applications
Fine, S., Singer, Y., & Tishby, N. (1998) · Machine Learning · doi:10.1023/a:1007469218079 · Cited by 906 · see citing works
Recurrent Network Models for Human Dynamics
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.494 · Cited by 904 · see citing works
Brand, M., & Hertzmann, A. (2000) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/344779.344865 · Cited by 704 · see citing works
Chen, C., Liu, Y., Kreiss, S., & Alahi, A. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794134 · Cited by 633 · see citing works
Learning Trajectory Dependencies for Human Motion Prediction
Mao, W., Liu, M.-M., Salzmann, M., & Li, H. (2019) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2019.00958 · Cited by 591 · see citing works
Neural Relational Inference for Interacting Systems
Kipf, T., Fetaya, E., Wang, K.-C. J., Welling, M., & Zemel, R. S. (2018) · UvA-DARE (University of Amsterdam) · Cited by 475 · see citing works
Graph networks as learnable physics engines for inference and control
Sanchez-Gonzalez, A., Heess, N., Springenberg, J. T., Merel, J., Riedmiller, M., Hadsell, R., & Battaglia, P. (2018) · arXiv · doi:10.48550/arxiv.1806.01242 · Cited by 387 · see citing works
Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction
Li, M., Chen, S.-H., Zhao, Y., Zhang, Y., Wang, Y.-F., & Tian, Q. (2020) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr42600.2020.00029 · Cited by 382 · see citing works
Self-citationConvolutional Sequence to Sequence Model for Human Dynamics
Li, C., Zhang, Z., Lee, W. S., & Lee, G. H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00548 · Cited by 364 · see citing works
Symbiotic Graph Neural Networks for 3D Skeleton-Based Human Action Recognition and Motion Prediction
Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., & Tian, Q. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3053765 · Cited by 243 · see citing works
Self-citationA Recurrent Neural Network that Learns to Count
Rodriguez, P. F., Wiles, J., & Elman, J. L. (1999) · Connection Science · doi:10.1080/095400999116340 · Cited by 240 · see citing works
Neural Relational Inference for Interacting SystemsPreprint
Kipf, T., Fetaya, E., Wang, K.-C. J., Welling, M., & Zemel, R. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04687 · Cited by 235 · see citing works
Learning Human Motion Models for Long-Term Predictions
Ghosh, P. S., Song, J., Aksan, E., & Hilliges, O. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00059 · Cited by 217 · see citing works
Learning Dynamic Relationships for 3D Human Motion Prediction
Cui, Q., Sun, H., & Yang, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00655 · Cited by 176 · see citing works
Probabilistic movement modeling for intention inference in human–robot interaction
Wang, Z., Mülling, K., Deisenroth, M. P., Ben Amor, H., Vogt, D., Schölkopf, B., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913478447 · Cited by 171 · see citing works
Self-citationAction-Agnostic Human Pose Forecasting
Chiu, H.-K., Adeli, E., Wang, B., Huang, D.-A., & Niebles, J. C. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00156 · Cited by 163 · see citing works
Kumar, R. (2017) · Apress eBooks · doi:10.1007/978-1-4842-3069-5_4 · Cited by 155 · see citing works
Unhelkar, V. V., Lasota, P. A., Tyroller, Q., Buhai, R.-D., Marceau, L., Deml, B., & Shah, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2812906 · Cited by 151 · see citing works
BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
Kundu, J. N., Gor, M., & Babu, R. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018553 · Cited by 128 · see citing works
Anticipating Many Futures: Online Human Motion Prediction and Generation for Human-Robot Interaction
Bütepage, J., Kjellström, H., & Kragić, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460651 · Cited by 110 · see citing works
Propagation Networks for Model-Based Control Under Partial Observation
Li, Y., Wu, J., Zhu, J.-Y., Tenenbaum, J. B., Torralba, A. B., & Tedrake, R. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793509 · Cited by 109 · see citing works
A Note on Over-Smoothing for Graph Neural NetworksPreprint
Cai, C., & Wang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13318 · Cited by 107 · see citing works
Self-citationHuman Motion Prediction via Learning Local Structure Representations and Temporal Dependencies
Guo, X. J., & Choi, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012580 · Cited by 106 · see citing works
Towards Efficient Human-Robot Collaboration With Robust Plan Recognition and Trajectory Prediction
Cheng, Y., Sun, L., Liu, C., et al. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2972874 · Cited by 100 · see citing works
A generative model for intention recognition and manipulation assistance in teleoperation
Tanwani, A. K., & Calinon, S. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202136 · Cited by 63 · see citing works
Efficient convolutional hierarchical autoencoder for human motion prediction
Li, Y., Wang, Z., Yang, X., Wang, M., Poiana, S. I., Chaudhry, E., & Zhang, J. (2019) · The Visual Computer · doi:10.1007/s00371-019-01692-9 · Cited by 56 · see citing works
Human Intention Inference Using Expectation-Maximization Algorithm With Online Model Learning
Ravichandar, H. C., & Dani, A. P. (2016) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2016.2624279 · Cited by 49 · see citing works
Kononenko, I., & Kukar, M. (2007) · Elsevier eBooks · doi:10.1533/9780857099440.59 · Cited by 45 · see citing works
Wang, Y., Ye, X., Yang, Y., et al. (2017) · doi:10.1109/humanoids.2017.8246890 · Cited by 37 · see citing works
Self-citationAIR-Act2Act: Human–human interaction dataset for teaching non-verbal social behaviors to robots
Ko, W.-R., Jang, M., Lee, J., et al. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364921990671 · Cited by 37 · see citing works
LSTM-based Network for Human Gait Stability Prediction in an Intelligent Robotic Rollator
Chalvatzaki, G., Koutras, P., Hadfield, J., et al. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793899 · Cited by 33 · see citing works
Self-citationChalvatzaki, G., Papageorgiou, X. S., Tzafestas, C. S., & Maragos, P. A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2800084 · Cited by 33 · see citing works
Self-citationMetric Learning from Poses for Temporal Clustering of Human Motion
López-Méndez, A., Gall, J., Casas, J. R., et al. (2012) · British Machine Vision Conference (BMVC) · doi:10.5244/c.26.49 · Cited by 31 · see citing works
Chalvatzaki, G., Papageorgiou, X. S., Maragos, P. A., et al. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2929996 · Cited by 26 · see citing works
Self-citationSymbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion PredictionPreprint
Li, M., Chen, S., Chen, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02212 · Cited by 13 · see citing works
A Convolutional Sequence to Sequence Model for Multimodal Dynamics Prediction in Ski Jumps
Dan, Z.-C., Eggert, C., Einfalt, M., Brehm, S., & Lienhart, R. (2018) · MMSports@MM · doi:10.1145/3265845.3265855 · Cited by 12 · see citing works
Ji, Z., Liu, Q., Xu, W., et al. (2020) · 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case48305.2020.9217003 · Cited by 10 · see citing works
Tekden, A. E., Erdem, A., Erdem, E., et al. (2019) · arXiv (Cornell University) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Chen, S., Peng, C., Yang, B., Lin, J., Zhou, L., Jiang, Z., Liu, Z., Liu, Y., & Tang, L. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00083-5 · Cited by 9 · see citing works
- Authors
- Li, Q., Chalvatzaki, G., Peters, J., & Wang, Y.
- Affiliations
- Central South University (CN)
- Technische Universität Darmstadt (DE)
- Venue
- 2021 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2021-05-30
- Pages
- 3197-3204
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W3205725086
- Cited by
- 29
- References
- 51
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 8 · 2023: 5 · 2022: 5
- FWCI
- 6.71
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- human motion prediction, human-robot interaction, encoder-decoder architecture, temporal dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
