The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
HP-GAN: Probabilistic 3D Human Motion Prediction via GAN
Open accessAbstract
Predicting and understanding human motion dynamics has many applications, such as motion synthesis, augmented reality, security, and autonomous vehicles. Due to the recent success of generative adversarial networks (GAN), there has been much interest in probabilistic estimation and synthetic data generation using deep neural network architectures and learning algorithms. We propose a novel sequence-to-sequence model for probabilistic human motion prediction, trained with a modified version of improved Wasserstein generative adversarial networks (WGAN-GP), in which we use a custom loss function designed for human motion prediction. Our model, which we call HP-GAN, learns a probability density function of future human poses conditioned on previous poses. It predicts multiple sequences of possible future human poses, each from the same input sequence but a different vector z drawn from a random distribution. Furthermore, to quantify the quality of the non-deterministic predictions, we simultaneously train a motion-quality-assessment model that learns the probability that a given skeleton sequence is a real human motion. We test our algorithm on two of the largest skeleton datasets: NTURGB-D and Human3.6M. We train our model on both single and multiple action types. Its predictive power for long-term motion estimation is demonstrated by generating multiple plausible futures of more than 30 frames from just 10 frames of input. We show that most sequences generated from the same input have more than 50% probabilities of being judged as a real human sequence. We published all the code used in this paper to https://github.com/ebarsoum/hpgan.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
Shahroudy, A., Liu, J., Ng, T.-T., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.115 · Cited by 3,148
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Sion, M. (1958) · Pacific Journal of Mathematics · doi:10.2140/pjm.1958.8.171 · Cited by 1,843
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2539 · Cited by 1,187
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133
Gaussian Process Dynamical Models for Human Motion
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1167 · Cited by 1,020
On Human Motion Prediction Using Recurrent Neural Networks
Martinez, J., Black, M. J., & Romero, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.497 · Cited by 984
Recurrent Network Models for Human Dynamics
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.494 · Cited by 860
Sequential Deep Learning for Human Action Recognition
Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., & Baskurt, A. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25446-8_4 · Cited by 782
End-to-End Learning of Action Detection from Frame Glimpses in Videos
Yeung, S., Russakovsky, O., Mori, G., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.293 · Cited by 585
Jointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.497 · Cited by 563
Gaussian Process Dynamical Models
Wang, J., Hertzmann, A., & Blei, D. M. (2005) · Cited by 438
Guiding the Long-Short Term Memory Model for Image Caption Generation
Jia, X., Gavves, E., Fernando, B., & Tuytelaars, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.277 · Cited by 426
Deep Representation Learning for Human Motion Prediction and Classification
Butepage, J., Black, M. J., Kragic, D., & Kjellstrom, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.173 · Cited by 406
Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities
Qi, G.-J. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01265-2 · Cited by 341
Learning Switching Linear Models of Human Motion
Pavlović, V., Rehg, J. M., & MacCormick, J. (2000) · Cited by 331
Implicit Probabilistic Models of Human Motion for Synthesis and Tracking
Sidenbladh, H., Black, M. J., & Sigal, L. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47969-4_52 · Cited by 314
Latent structured models for human pose estimation
Ionescu, C., Li, F., & Sminchisescu, C. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126500 · Cited by 258
Efficient Nonlinear Markov Models for Human Motion
Lehrmann, A. M., Gehler, P. V., & Nowozin, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.171 · Cited by 187
Learning Spatio-Temporal Structure from RGB-D Videos for Human Activity Detection and Anticipation
Koppula, H. S., & Saxena, A. (2013) · International Conference on Machine Learning · Cited by 169
Perception of human motion with different geometric models
Hodgins, J. K., O'Brien, J. F., & Tumblin, J. (1998) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.765325 · Cited by 161
A data-driven approach to quantifying natural human motion
Ren, L., Patrick, A., Efros, A. A., Hodgins, J. K., & Rehg, J. M. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073316 · Cited by 143
Generative Adversarial Nets from a Density Ratio Estimation PerspectivePreprint
Uehara, M., Sato, I., Suzuki, M., Nakayama, K., & Matsuo, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02920 · Cited by 106
Recurrent Network Models for Human DynamicsPreprint
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.00271 · Cited by 89
Stabilizing GAN Training with Multiple Random ProjectionsPreprint
Neyshabur, B., Bhojanapalli, S., & Chakrabarti, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07831 · Cited by 86
Obscuring length changes during animated motion
Harrison, J., Rensink, R. A., & van de Panne, M. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015761 · Cited by 62
Video Imagination from a Single Image with Transformation Generation
Chen, B., Wang, W., & Wang, J. (2017) · Thematic Workshops of ACM Multimedia · doi:10.1145/3126686.3126737 · Cited by 50
Video Description Generation Incorporating Spatio-Temporal Features and a Soft-Attention MechanismPreprint
Li, Y., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. C. (2015) · arXiv (Cornell University) · Cited by 43
3D Human Motion Editing and Synthesis: A Survey
Wang, X., Chen, Q., & Wang, W. (2014) · Computational and Mathematical Methods in Medicine · doi:10.1155/2014/104535 · Cited by 38
Cited by
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437
Self-citationGenerating Human Motion from Textual Descriptions with Discrete Representations
Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., & Ying, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01415 · Cited by 328
Adversarial Geometry-Aware Human Motion Prediction
Gui, L.-Y., Wang, Y.-X., Liang, X., & Moura, J. M. F. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_48 · Cited by 276
Harvey, F. G., Yurick, M., Nowrouzezahrai, D., & Pal, C. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392480 · Cited by 266
Weaponized AI for cyber attacks
Yamin, M. M., Ullah, M., Ullah, H., & Katt, B. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2020.102722 · Cited by 154
Metadata
- Authors
- Barsoum, E., Kender, J., & Liu, Z.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 1499-149909
- OpenAlex
- W2769460278
- Cited by
- 322
- References
- 38
- FWCI
- 37.87
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.