A Neural Temporal Model for Human Motion Prediction
Open accessAbstract
We propose novel neural temporal models for predicting and synthesizing human motion, achieving state-of-the-art in modeling long-term motion trajectories while being competitive with prior work in short-term prediction and requiring significantly less computation. Key aspects of our proposed system include: 1) a novel, two-level processing architecture that aids in generating planned trajectories, 2) a simple set of easily computable features that integrate derivative information, and 3) a novel multi-objective loss function that helps the model to slowly progress from simple next-step prediction to the harder task of multi-step, closed-loop prediction. Our results demonstrate that these innovations improve the modeling of long-term motion trajectories. Finally, we propose a novel metric, called Normalized Power Spectrum Similarity (NPSS), to evaluate the long-term predictive ability of motion synthesis models, complementing the popular mean-squared error (MSE) measure of Euler joint angles over time. We conduct a user study to determine if the proposed NPSS correlates with human evaluation of long-term motion more strongly than MSE and find that it indeed does. We release code and additional results (visualizations) for this paper at: https://github.com/cr7anand/neural_temporal_models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1× · see citing works
“With the modern successes of artificial neural networks [15] in application domains ranging from computer vision [13] to machine translation [1] and language modeling [18], many current models of motion are becoming increasingly based on neural architectures.”
BackgroundHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 8× · see citing works
“The LSTM-3LR consists of 3 layers of 1000 Long Short-Term Memory (LSTM) units whereas the ERD model uses 2 layers of 1000 LSTM units and non-linear multilayer feedforward networks for encoding and decoding.”
MethodInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1× · see citing works
“Recently, [17] proposed simple but hard-to-beat baselines on short-term motion prediction as well as a 1-layer seq2seq model [26] with 1024 Gated Recurrent Unit (GRU) units and a linear output decoder for short-term and long-term motion prediction.”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1× · see citing works
“With the modern successes of artificial neural networks [15] in application domains ranging from computer vision [13] to machine translation [1] and language modeling [18], many current models of motion are becoming increasingly based on neural architectures.”
BackgroundEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 2× · see citing works
“The cell update equations for the top-level GRU remain the same as described in [3] except that we note its non-state input is the action label (which remains fixed over the length of the sequence).”
BackgroundMethodThe Earth Mover's Distance as a Metric for Image Retrieval
Rubner, Y., Tomasi, C., & Guibas, L. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026543900054 · Cited by 5,547 · see citing works
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1× · see citing works
“6m) dataset [9], which is currently the largest publicly available motion capture (mocap) database.”
BackgroundRecurrent Neural Network RegularizationPreprint
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv.org · doi:10.48550/arxiv.1409.2329 · Cited by 3,086 · cited 1× · see citing works
“Drop-out [30, 20], with probability of 0.3, was applied only to the Body-RNN, which was further modified to use skip connections that connect input units to output units, as in [17].”
MethodInfluentialStructural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133 · cited 14× · see citing works
“Staying consistent with previous work on human motion synthesis [4, 10], we use the Human 3.”
BackgroundMethodInfluentialGaussian Process Dynamical Models for Human Motion
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1167 · Cited by 1,024 · see citing works
On Human Motion Prediction Using Recurrent Neural Networks
Martínez, J., Black, M. J., & Romero, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.497 · Cited by 1,019 · cited 15× · see citing works
“Therefore, as noted in prior work [4, 10, 17], the use of MSE as an evaluation metric is not appropriate in the long-term task.”
BackgroundMethodResultInfluentialRecurrent Network Models for Human Dynamics
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.494 · Cited by 904 · see citing works
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Serban, I. V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A. C., & Bengio, Y. (2017) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v31i1.10983 · Cited by 734 · cited 1× · see citing works
“The motivation behind the VTLN-RNN structure was to hierarchically decompose the motion synthesis problem into a two-level process, much as has been successfully done in neural-based dialogue modeling [23, 22].”
MethodVariational Dropout and the Local Reparameterization TrickPreprint
Kingma, D. P., Salimans, T., & Welling, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02557 · Cited by 708 · cited 1× · see citing works
“In order to sample the initial hidden state of the top-level noise process, we first structure it to work like a multivariate Gaussian distribution, drawing inspiration from the reparameterization trick [11] and the adaptive noise scheme proposed in [8].”
MethodModeling Human Motion Using Binary Latent Variables
Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0173 · Cited by 708 · cited 1× · see citing works
“Pose is represented as an exponential map of each joint (refer to [27] for further details).”
BackgroundKovar, L., Gleicher, M. L., & Pighin, F. P. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566605 · Cited by 596 · see citing works
Dropout improves Recurrent Neural Networks for Handwriting RecognitionPreprint
(2016) · Cited by 514 · see citing works
A global human walking model with real-time kinematic personification
Boulic, R., Magnenat‐Thalmann, N., & Thalmann, D. (1990) · The Visual Computer · doi:10.1007/bf01901021 · Cited by 504 · cited 1× · see citing works
“Models successful in these sub-tasks are also valuable for human gait analysis, studies in the kinematics of human motion, and in human-computer interaction applications [2, 25].”
BackgroundProfessor Forcing: A New Algorithm for Training Recurrent NetworksPreprint
Lamb, A., Goyal, A., Zhang, Y., Zhang, S., Courville, A. C., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09038 · Cited by 402 · cited 1× · see citing works
“Professor Forcing [14] addresses this issue by using an adversarial training regime to ensure the hidden states of the RNN are constrained to be similar during train and test time.”
MethodLearning Switching Linear Models of Human Motion
Pavlović, V., Rehg, J. M., & MacCormick, J. (2000) · Cited by 320 · see citing works
Implicit Probabilistic Models of Human Motion for Synthesis and Tracking
Sidenbladh, H., Black, M. J., & Sigal, L. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47969-4_52 · Cited by 318 · see citing works
Fast contour matching using approximate Earth mover's distance
Grauman, K., & Darrell, T. J. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315035 · Cited by 219 · see citing works
Learning Human Motion Models for Long-Term Predictions
Ghosh, P. S., Song, J., Aksan, E., & Hilliges, O. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00059 · Cited by 217 · cited 1× · see citing works
“More recently, [5] proposed a model that couples a de-noising autoencoder and a LSTM-3LR network to alleviate drifting for long-term motion synthesis.”
BackgroundContinuous character control with low-dimensional embeddings
Levine, S., Wang, J. M., Haraux, A., Popović, Z., & Koltun, V. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185524 · Cited by 170 · cited 1× · see citing works
“Short-term models are useful in applications of motion tracking while long-term models are useful as motion generation tools for computer graphics [24, 16, 12].”
BackgroundAuto-Conditioned Recurrent Networks for Extended Complex Human Motion SynthesisPreprint
Li, Z., Zhou, Y., Xiao, S., He, C., Huang, Z., & Li, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05363 · Cited by 150 · see citing works
Continuous body and hand gesture recognition for natural human-computer interaction
Song, Y., Demirdjian, D., & Davis, R. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2133366.2133371 · Cited by 149 · cited 1× · see citing works
“Models successful in these sub-tasks are also valuable for human gait analysis, studies in the kinematics of human motion, and in human-computer interaction applications [2, 25].”
BackgroundCorrection to "Gaussian Process Dynamical Models for Human Motion" [Feb 08 283-298]
Wang, J. M., Fleet, D. J., & Hertzmann, A. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.91 · Cited by 122 · see citing works
Dropout Improves Recurrent Neural Networks for Handwriting Recognition
Pham, V., Bluche, T., Kermorvant, C., & Louradour, J. (2014) · International Conference on Frontiers in Handwriting Recognition · doi:10.1109/icfhr.2014.55 · Cited by 72 · cited 1× · see citing works
“Drop-out [30, 20], with probability of 0.3, was applied only to the Body-RNN, which was further modified to use skip connections that connect input units to output units, as in [17].”
MethodInfluentialLearning Simpler Language Models with the Differential State Framework
Ororbia, A. G., Mikolov, T., & Reitter, D. (2017) · Neural Computation · doi:10.1162/neco_a_01017 · Cited by 62 · see citing works
Self-citationBiologically Motivated Algorithms for Propagating Local Target Representations
Ororbia, A. G., & Mali, A. (2019) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v33i01.33014651 · Cited by 55 · cited 2× · see citing works
“m-level RNN. Combined, the 2 RNNs have fewer parameters than prior motion deep learning motion synthesis models. The top-levelRNN is meant to serveas a learnable noise process inspired by the work of [7], which runs backwards in time, starting from a sampled initial hidden state (zφ) and is conditioned on the one-hot…”
MethodSelf-citationContinuous character control with low-dimensional embeddings
Levine, S., Wang, J. M., Haraux, A., et al. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2335379 · Cited by 39 · see citing works
Piecewise Latent Variables for Neural Variational Text Processing
Serban, I. V., Ororbia, A. G., Pineau, J., et al. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1043 · Cited by 30 · see citing works
Self-citationPiecewise Latent Variables for Neural Variational Text Processing
Serban, I. V., Ororbia, A. G., Pineau, J., et al. (2017) · doi:10.18653/v1/w17-4308 · Cited by 11 · cited 1× · see citing works
“The motivation behind the VTLN-RNN structure was to hierarchically decompose the motion synthesis problem into a two-level process, much as has been successfully done in neural-based dialogue modeling [23, 22].”
MethodSelf-citation
Flagged first, then by citations
Research works printed but not linked: 1
- tieleman, 2012, Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude, COURSERA Neural Networks for Machine Learning
Cited by
Sorted by citations
Multi-Sensor Fusion in Automated Driving: A Survey
Wang, Z., Wu, Y., & Niu, Q. (2020) · IEEE Access · doi:10.1109/access.2019.2962554 · Cited by 526 · see citing works
“Pedestrian detection is a significant portion of AD, and relevant researches have carried out in recent years [133], [134].”
BackgroundDynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction
Li, M., Chen, S.-H., Zhao, Y., Zhang, Y., Wang, Y.-F., & Tian, Q. (2020) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr42600.2020.00029 · Cited by 382 · see citing works
“Many methods, including the conventional state-based methods [25, 44, 38, 37, 36] and deep-network-based methods [9, 32, 10, 7, 12, 14, 11, 33, 43], have been proposed to Figure 1.”
BackgroundMethodHistory Repeats Itself: Human Motion Prediction via Motion Attention
Mao, W., Liu, M., & Salzmann, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_28 · Cited by 322 · see citing works
“Following previous works [7,17,20,22,25], we evaluate our method on Human3.”
MethodHarvey, F. G., Yurick, M., Nowrouzezahrai, D., & Pal, C. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392480 · Cited by 269 · see citing works
“On another front, many auto-regressive deep learning methods that leverage high quality MOCAP for motion prediction have recently been proposed [Barsoum et al. 2018; Chiu et al. 2019; Fragkiadaki et al. 2015; Gopalakrishnan et al. 2019; Jain et al. 2016; Martinez et al. 2017; Pavllo et al. 2019].”
MethodResultInfluentialDLow: Diversifying Latent Flows for Diverse Human Motion Prediction
Yuan, Y., & Kitani, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_20 · Cited by 259 · see citing works
“sample diversity and accuracy. 2 Related Work Human Motion Prediction. Most previous work takes a deterministic approach to modeling human motion and regress a single future motion from past 3D poses [17,34,9,45,18,52,57,13,22,1,69,51] or video frames [10,77,75]. While these approaches are able to predict the most lik…”
BackgroundSymbiotic Graph Neural Networks for 3D Skeleton-Based Human Action Recognition and Motion Prediction
Li, M., Chen, S., Chen, X., Zhang, Y., Wang, Y., & Tian, Q. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3053765 · Cited by 243 · see citing works
“98 VGRU-r1 [57] 0.”
PhysDiff: Physics-Guided Human Motion Diffusion Model
Yuan, Y., Song, J., Iqbal, U., Vahdat, A., & Kautz, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01467 · Cited by 246 · see citing works
“Early work on motion generation adopts deterministic human motion modeling which only generates a single motion [17, 38, 18, 51, 56, 20, 2, 81, 50, 86].”
BackgroundHuman Motion Prediction via Spatio-Temporal Inpainting
Hernandez, A., Gall, J., & Moreno, F. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00723 · Cited by 196 · see citing works
“In [10] a metric based in power spectrum is proposed.”
BackgroundMa, T., Nie, Y.-W., Long, C., Zhang, Q., & Li, G. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00633 · Cited by 170 · see citing works
DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer
Li, B., Zhao, Y., Shi, Z., & Sheng, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20014 · Cited by 143 · see citing works
“We just follow its official implementation (Gopalakrishnan et al. 2019) and compute NPSS in the joint motion space R T × N × 7 ( 4 for joint rotation represented as quaternion and 3 for joint position).”
MethodSpatio-Temporal Gating-Adjacency GCN for Human Motion Prediction
Zhong, C.-Y., Hu, L., Zhang, Z., Ye, Y.-J., & Xia, S. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00634 · Cited by 131 · see citing works
Music2Dance: DanceNet for Music-Driven Dance Generation
Zhuang, W., Wang, C., Chai, J., Wang, Y., Shao, M., & Xia, S. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3485664 · Cited by 134 · see citing works
“We use the prediction accuracy to evaluate the final results, and our method can obtain better results than the convolution RNN and baseline, as shown in Table 3.”
BackgroundInfluentialTEACH: Temporal Action Composition for 3D Humans
Athanasiou, N., Petrovich, M., Black, M. J., & Varol, G. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00053 · Cited by 141 · see citing works
“Due to the complex spatio-temporal nature of the input data, commonly modeled using skeletal representations, most recent work relies on autoregressive neural models [8, 12, 27, 29].”
MethodGLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Yuan, Y., Iqbal, U., Molchanov, P. A., Kitani, K., & Kautz, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01076 · Cited by 133 · see citing works
“Extensive research has studied 3D human dynamics for various tasks including motion prediction and synthesis [2,4,8,20,21,27,39,60,67,79,81,100, 107,109–111].”
BackgroundWe are More than Our Joints: Predicting how 3D Bodies Move
Zhang, Y., Black, M. J., & Tang, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00338 · Cited by 129 · see citing works
“This task has been extensively studied [5, 6, 10, 13, 15, 16, 17, 18, 28, 29, 35, 36, 40, 41, 47, 49, 60].”
TrajectoryCNN: A New Spatio-Temporal Feature Learning Network for Human Motion Prediction
Liu, X., Yin, J., Liu, J., Ding, P., Liu, J., & Liu, H. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3021409 · Cited by 122 · see citing works
“(1) RNN models [11], [12], [14] are especially powerful in processing short-term temporal information while having an inherent weakness in spatial modeling.”
BackgroundMethodGOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping
Taheri, O., Choutas, V., Black, M. J., & Tzionas, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01291 · Cited by 118 · see citing works
“Early deep-learning methods employ RNNs [12,15,44], but they struggle with discontinuities between the observed and predicted poses, as well as with long-range spatio-temporal relations.”
BackgroundHuman Motion Generation: A Survey
Zhu, W., Ma, X., Ro, D., Ci, H., Zhang, J., Shi, J., Gao, F., Tian, Q. C., & Wang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3330935 · Cited by 118 · see citing works
Efficient human motion prediction using temporal convolutional generative adversarial network
Cui, Q., Sun, H., Kong, Y., Zhang, X., & Li, Y. (2020) · Information Sciences · doi:10.1016/j.ins.2020.08.123 · Cited by 93 · see citing works
Multi-level Motion Attention for Human Motion Prediction
Mao, W., Liu, M., Salzmann, M., & Li, H. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01483-7 · Cited by 90 · see citing works
“As such, they have been widely employed for human motion prediction (Fragkiadaki et al., 2015; Jain et al., 2016; Martinez et al., 2017; Gopalakrishnan et al., 2019).”
MethodInfluentialGenerating Smooth Pose Sequences for Diverse Human Motion Prediction
Mao, W., Liu, M., & Salzmann, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01306 · Cited by 88 · see citing works
Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction
Li, M., Chen, S., Zhao, Y., Zhang, Y., Wang, Y., & Tian, Q. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3108708 · Cited by 84 · see citing works
Dynamic Dense Graph Convolutional Network for Skeleton-Based Human Motion Prediction
Wang, X., Zhang, W., Wang, C., Gao, Y., & Liu, M.-Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3334954 · Cited by 73 · see citing works
“To handle this problem, most early work employs Recurrent Neural Networks (RNNs) to address the sequential nature of human motion [12], [13], [14], [15], [16], [17], [18] but the difficulty of training them poses a challenge.”
Background3D human motion prediction: A survey
Lyu, K., Chen, H., Liu, Z., Zhang, B., & Wang, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.045 · Cited by 81 · see citing works
Towards Accurate 3D Human Motion Prediction from Incomplete Observations
Cui, Q., & Sun, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00477 · Cited by 81 · see citing works
“[20] construct two-level processing to help generate the planned motion trajectory.”
BackgroundA Survey on Deep Learning for Skeleton‐Based Human Animation
Mourot, L., Hoyet, L., Clerc, F., Schnitzler, F., & Pierre Hellier, J. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14426 · Cited by 88 · see citing works
Motion Prediction using Trajectory Cues
Liu, Z., & Wu, S. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.01305 · Cited by 64 · see citing works
“One line of work [8, 12, 13, 22] utilized recur-13299 rent neural networks (RNNs) and various RNN variants to model motion contexts.”
MethodPVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction
Wang, H., Dong, J., Cheng, B., & Feng, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3089380 · Cited by 55 · see citing works
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations
Ororbia, A. G., Mali, A., Giles, C. L., & Kifer, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2953622 · Cited by 60 · see citing works
“These tasks range from those in statistical machine translation [6], to language modeling and text processing [7], [8], [9], to long and short-term human motion generation [10], to speech recognition [11].”
BackgroundSelf-citationMotion In-Betweening via Two-Stage Transformers
Qin, J., Zheng, Y., & Zhou, K. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555454 · Cited by 85 · see citing works
“The Normalized Power Spectrum Similarity (NPSS), proposed by [Gopalakrishnan et al. 2019], evaluates angular differences between predicted motion and ground truth on the frequency domain.”
MethodPose Forecasting in Industrial Human-Robot Collaboration
Sampieri, A., di Melendugno, G. M. D., Avogaro, A., Cunico, F., Setti, F., Skenderi, G., Cristani, M., & Galasso, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_4 · Cited by 50 · see citing works
The Wanderings of Odysseus in 3D Scenes
Zhang, Y., & Tang, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01983 · Cited by 41 · see citing works
RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting
Li, J., Yang, F., Ma, H., Malla, S., Tomizuka, M., & Choi, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01579 · Cited by 41 · see citing works
“• Human motions: Recurrent neural networks are widely utilized in human motion forecasting [11, 16, 22].”
BackgroundCollaborative Multi-Dynamic Pattern Modeling for Human Motion Prediction
Tang, J., Zhang, J., Ding, R., Gu, B., & Yin, J.-Q. (2023) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2023.3239322 · Cited by 39 · see citing works
“Thus many works [7], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21] implement RNN to model the temporal sequence effectively.”
BackgroundZheng, J., Zheng, Q., Fang, L., Liu, Y., & Yi, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00064 · Cited by 38 · see citing works
“Human motion synthesis, including motion prediction, interpolation, and completion, has attracted many interests these years [1,2,4,8,9,12,14–16,20,21,26,30,37,38,42– 44].”
BackgroundWeakly-supervised Action Transition Learning for Stochastic Human Motion Prediction
Mao, W., Liu, M., & Salzmann, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00798 · Cited by 36 · see citing works
“The only exceptions that incorporate action information are the early works of [13,15,16,21,38].”
BackgroundMethodDeep Learning for Vision-based Prediction: A SurveyPreprint
Rasouli, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00095 · Cited by 44 · see citing works
“Precision+Recall RMSE = √√√√ 1 n n∑ i=1 (yi − ỹi) Let p(r) be the precision-recall curve.”
MethodInfluentialBhatt, N. P., Khajepour, A., & Hashemi, E. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3243004 · Cited by 34 · see citing works
“There are two main challenges with trajectory prediction of objects namely: (i) anticipating dependence of past trajectories of objects on their future trajectory [9], [10], [11] and (ii) incorporating the effect of social interaction between objects in future trajectory estimates [12].”
BackgroundSingle-Shot Motion Completion with TransformerPreprint
Duan, Y., Shi, T., Zou, Z., Lin, Y., Qian, Z., Zhang, B., & Yuan, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.00776 · Cited by 56 · see citing works
“3 The NPSS, proposed by Gopalakrishnan [12], is a variant of L2Q, which computes the Normalized Power Spectrum Similarity and is based on angular frequency distance be-tween the prediction and the groundtruth.”
BackgroundPACE: Human and Camera Motion Estimation from in-the-wild Videos
Kocabas, M., Yuan, Y., Molchanov, P. A., Guo, Y., Black, M. J., Hilliges, O., Kautz, J., & Iqbal, U. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00103 · Cited by 34 · see citing works
“Along this line, a recent work [105] obtains initial camera trajectories with SLAM and optimizes the scale of the camera trajectories using a human motion prior [85].”
MethodA Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction
Men, Q.-H., Ho, E. S. L., Shum, H. P. H., & Leung, H. (2021) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2020.3038145 · Cited by 32 · see citing works
“…state-of-the-art multi-label algorithms related to our network architecture under two types of baselines, which are RNN or CNN-based models: RRNN [8], 3D-PFNet [10], RMA [32], TP-RNN [17], VGRU [10], QuaterNet [36], BiHMP-GAN [16], and ConvSeq2seq [15]; GCN-based models: LDR [29] and DMGNN [28].”
BackgroundWearable Motion Capture: Reconstructing and Predicting 3D Human Poses From Wearable Sensors
Moniruzzaman, M., Yin, Z., Hossain, M. S. B., Choi, H. Y., & Guo, Z. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3311448 · Cited by 29 · see citing works
“Recently, some deep-learning-based algorithms introduced recurrent networks [40], [41], [42], [43], [44] and feed-forward networks [45], [46], [47] to predict future 3D poses.”
BackgroundTemporal consistency two-stream CNN for human motion prediction
Tang, J. H., Zhang, J., & Yin, J. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.011 · Cited by 30 · see citing works
A Unified Framework for Real Time Motion Completion
Duan, Y., Lin, Y., Zou, Z., Yuan, Y. G., Qian, Z., & Zhang, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20368 · Cited by 28 · see citing works
Social Diffusion: Long-term Multiple Human Motion Anticipation
Tanke, J., Zhang, L., Zhao, A. T., Tang, C., Cai, Y., Wang, L., Wu, P.-C., Gall, J., & Keskin, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00880 · Cited by 28 · see citing works
“Current state-of-the-art multi-person motion anticipation methods [3, 69] calculate the Mean Per Joint Po-sitional Error (MPJPE) using the ground-truth sequence, which is only meaningful for short time horizons of around one second [6, 21, 53, 61, 71].”
MethodInteractive Character Control with Auto-Regressive Motion Diffusion Models
Shi, Y., Wang, J., Jiang, X., Lin, B., Dai, B., & Peng, X. B. (2024) · ACM Transactions on Graphics · doi:10.1145/3658140 · Cited by 29 · see citing works
“…for auto-regressive motion generation have often used CNNs, LSTMs, and MLPs to sequentially predict the next frame conditioned on previously generated frames [Aksan et al. 2019; Fragkiadaki et al. 2015; Gopalakrishnan et al. 2019; Li et al. 2017; Martinez et al. 2017; Pavllo et al. 2018].”
Human Motion Prediction via Spatio-Temporal InpaintingPreprint
Ruiz, A. H., Gall, J., & Moreno-Noguer, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05478 · Cited by 26 · see citing works
“In [10] a metric based in power spectrum is proposed.”
BackgroundMotionAug: Augmentation with Physical Correction for Human Motion Prediction
Maeda, T., & Ukita, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00632 · Cited by 25 · see citing works
“Most approaches [6,12,13,15,21,32,35,45,51] are built upon Recurrent Neural Networks (RNN) to model sequenceto-sequence relationships between past and future motions.”
Few-shot Human Motion Prediction via Learning Novel Motion Dynamics
Zang, C., Pei, M., & Kong, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/118 · Cited by 23 · see citing works
“Many researchers extract the guidance information of motion dynamics from different cues, such as speed [ Martinez et al. , 2017 ] , incorporating derivative [ Gopalakrishnan et al. , 2019 ] , and smooth trajectory [ Wei et al. , 2019 ] .”
BackgroundMethodGenerative model‐enhanced human motion prediction
Bourached, A., Griffiths, R., Gray, R., Jha, A., & Nachev, P. (2022) · Applied AI Letters · doi:10.1002/ail2.63 · Cited by 23 · see citing works
“Historically, sequence-to-sequence prediction using Recurrent Neural Networks (RNNs) have been the de facto standard for human motion prediction [22, 33, 24, 34, 35, 27].”
BackgroundComputer Vision Enabled Drowning Detection System
Handalage, U., Nikapotha, N., Subasinghe, C., et al. (2021) · International Conference on Advancements in Computing (ICAC) · doi:10.1109/icac54203.2021.9671126 · Cited by 23 · see citing works
Skeleton-based Action Recognition for Human-Robot Interaction using Self-Attention Mechanism
Bandi, C., & Thomas, U. (2021) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg52635.2021.9666948 · Cited by 23 · see citing works
Forecasting Characteristic 3D Poses of Human Actions
Diller, C., Funkhouser, T. A., & Dai, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01545 · Cited by 24 · see citing works
Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints
Li, J.-C., Shi, X., Chen, F., Stroud, J. C., Zhang, Z., Lan, T., Mao, J.-H., Kang, J., Refaat, K. S., Yang, W., et al. (2023) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra48891.2023.10160273 · Cited by 30 · see citing works
“Human Keypoints Encoding: Extracting informative features from human keypoints is crucial for human action recognition and motion prediction [38]–[46].”
Incorporating Physics Principles for Precise Human Motion Prediction
Zhang, Y., Kephart, J. O., & Ji, Q. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00605 · Cited by 24 · see citing works
“Various deep models have been proposed to model motion sequences, such as those based on Recurrent Neural Networks (RNN) [10,16,19,47], Convolutional Neural Networks (CNN) [4, 9, 32, 34, 56], and Transformers [3, 39, 48].”
MethodTask-Oriented Human-Object Interactions Generation with Implicit Neural Representations
Li, Q., Wang, J. B., Loy, C. C., & Dai, B. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00301 · Cited by 23 · see citing works
“Human motion synthesis has become a prevailing research topic in recent years and has drawn attention from both computer vision and computer graphics [23, 8, 24, 15, 16, 3, 22, 17, 28, 41].”
BackgroundKSOF: Leveraging kinematics and spatio-temporal optimal fusion for human motion prediction
Ding, R., Qu, K., & Tang, J. H. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111206 · Cited by 24 · see citing works
Skeleton-Based Human Motion Prediction With Privileged Supervision
Dong, M., & Xu, C. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3166861 · Cited by 22 · see citing works
“Following [26], we visualized some sequences with different activities from Human 3.6M and compared generated motion sequences of proposed PS with other current state-of-the-art algorithms, such as Traj-GCN and DMGNN [47], [49], as shown in Fig.”
BackgroundMethodInfluentialPredicting human poses via recurrent attention network
Tang, J., Wang, J., & Hu, J. (2023) · Visual Intelligence · doi:10.1007/s44267-023-00020-z · Cited by 22 · see citing works
Joint-Relation Transformer for Multi-Person Motion Prediction
Xu, Q., Mao, W., Gong, J., Xu, C., Chen, S., Xie, W., Zhang, Y., & Wang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00900 · Cited by 23 · see citing works
“With the development of deep learning, many RNN-based networks [11, 20, 14, 36, 16, 7, 30, 26] have been developed and achieve great success.”
BackgroundMotion In-Betweening via Deep -Interpolator
Oreshkin, B. N., Valkanas, A., Harvey, F. G., Ménard, L.-S., Bocquelet, F., & Coates, M. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3309107 · Cited by 21 · see citing works
“We consider L2Q (the global quaternion squared loss (6)), L2P (the global position squared loss (7)) and NPSS (the normalized power spectrum similarity score) [17].”
MethodPoseBERT: A Generic Transformer Module for Temporal 3D Human Modeling
Baradel, F., Brégier, R., Groueix, T., Weinzaepfel, P., Kalantidis, Y., & Rogez, G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3216899 · Cited by 19 · see citing works
“For pose sequence generation, previous methods either condition on the beginning of a sequence of pose [46], [47], [48], [49] or on some predefined labels like human actions [50], [51], [52].”
MethodLarge Motion Model for Unified Multi-modal Motion Generation
Zhang, M., Jin, D., Gu, C., Hong, F., Cai, Z., Huang, J., Zhang, C., Guo, X., Yang, L., He, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_23 · Cited by 21 · see citing works
Genre-Conditioned Long-Term 3D Dance Generation Driven by Music
Huang, Y., Zhang, J., Liu, S., Bao, Q., Zeng, D., Chen, Z., & Liu, W. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747838 · Cited by 18 · see citing works
Chen, L., Liu, R., Yang, X. S., Zhou, D., Zhang, Q., & Wei, X. (2022) · Visual Computing for Industry Biomedicine and Art · doi:10.1186/s42492-022-00112-5 · Cited by 18 · see citing works
“[17] used derivative information as a computational feature in a neuro-temporal model with a two-level processing architecture containing a top-level and bottom-level RNN.”
MethodMo, C., Hu, K., Long, C., & Wang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01335 · Cited by 18 · see citing works
“In addition, we apply the Normalised Power Spectrum Similarity (NPSS) [12] metric to measure visual similarities between the estimated and actual motion outputs.”
MethodInfluentialTrajevae: Controllable Human Motion Generation from TrajectoriesPreprint
Kania, K., Kowalski, M., & Trzcinski, T. P. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4092912 · Cited by 18 · see citing works
“…In recent years, multiple methods were proposed for predicting a single future motion based on a corresponding past sequence of poses [2,3,10,12,18,20–22, 40 , 41 , 45– 47, 51, 53, Stochastic motion prediction To model the distribution of possible motions, recent works [4, 23, 25, 31, 35, 38, 43,…”
MethodMusic2Dance: Music-driven Dance Generation using WaveNet.Preprint
Zhuang, W., Wang, C., Xia, S., Chai, J., & Wang, Y. (2020) · arXiv (Cornell University) · Cited by 17 · see citing works
Virtual Instrument Performances (VIP): A Comprehensive Review
Kyriakou, T., Crespo, M. Á. D. L. C., Panayiotou, A., Chrysanthou, Y., Charalambous, P., & Aristidou, A. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15065 · Cited by 22 · see citing works
HMP: Hand Motion Priors for Pose and Shape Estimation from Video
Duran, E., Kocabas, M., Choutas, V., Fan, Z., & Black, M. J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00623 · Cited by 15 · see citing works
“We first reflect left hand articulations in the dataset for data augmentation.”
MethodIntention-based Long-Term Human Motion Anticipation
Tanke, J., Zaveri, C., & Gall, J. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00069 · Cited by 14 · see citing works
“The negative correlation is due to the competitive NPSS of Grammar [14] although the generated motions are perceived as unrealistic by humans.”
InfluentialZhai, D., Yan, Z., & Xia, Y. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3301657 · Cited by 13 · see citing works
“Because of the temporal nature of human motion, the most common trend is using recurrent neural networks (RNNs) [5], [6], [7], [8], [9], [10] or Transformers [11], [12].”
BackgroundAMD: Autoregressive Motion Diffusion
Han, B., Peng, H., Dong, M., Ren, Y., Shen, Y. L., & Xu, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.27973 · Cited by 15 · see citing works
Image-Guided Tethering Spine Surgery With Outcome Prediction Using Spatio-Temporal Dynamic Networks
Mandel, W., Oulbacha, R., Roy‐Beaudry, M., et al. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3030741 · Cited by 13 · see citing works
“supplanted classical statistical learning methods [16], where”
Spatio-Temporal Articulation & Coordination Co-attention Graph Network for human motion prediction
Zhu, S., Chen, J., & Su, Y. R. (2024) · Signal Processing · doi:10.1016/j.sigpro.2024.109551 · Cited by 12 · see citing works
Pięta, P., Jegierski, H., Babiuch, P., Jegierski, M., Płaza, M., Łukawski, G., Deniziak, S. R., Jasiński, A., Opałka, J., Wȩgrzyn, P., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3424930 · Cited by 13 · see citing works
“The complexity and variety of human movements means that systems must be able to understand and correctly interpret a wide range of gestures and body movements [31], [32].”
BackgroundHuman Motion Prediction Via Pattern Completion in Latent Representation Space
Xu, Y. T., Li, Y., & Meger, D. (2019) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv.2019.00016 · Cited by 12 · see citing works
“We cite the results from the most relevant works to compare with our method, which are Res-Seq2Seq [6], the model by Tang et al. [11], VGRU-rl [21] and AGED [8] which is the current state of the art.”
BackgroundMethodInfluentialDeep Learning for 3D Human Motion Prediction: State-of-the-Art and Future Trends
Marchellus, M., & Park, I. K. (2022) · IEEE Access · doi:10.1109/access.2022.3163269 · Cited by 12 · see citing works
“[9] proposed a new architecture, the verso-time label noise RNN (VTLN-RNN), to tackle both short-and long-term prediction problems simultaneously alongside a more competent long-term evaluation metric, which is the normalized power spectrum similarity (NPSS).”
BackgroundSpatio-Temporal Branching for Motion Prediction using Motion Increments
Wang, J., Zhou, Y., Qiang, W., Ba, Y., Su, B., & Wen, J.-R. (2023) · Proceedings of the 31st ACM International Conference on Multimedia · doi:10.1145/3581783.3612330 · Cited by 11 · see citing works
“RNNs [6, 9, 11, 13, 27] have been used to model the temporal dynamics of human motion, while CNNs [2, 7, 22] have been used to model the spatial information.”
MethodVelocity-to-velocity human motion forecasting
Wang, H., Wang, L., Feng, J., & Zhou, D. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108424 · Cited by 11 · see citing works
Fusion learning-based recurrent neural network for human motion prediction
Guo, C., Liu, R., Che, C., Zhou, D., Zhang, Q., & Wei, X. (2022) · Intelligent Service Robotics · doi:10.1007/s11370-021-00403-5 · Cited by 11 · see citing works
“followed the previous work [14, 25] to fix the coefficient α to 1.”
MethodDanceNet3D: Music Based Dance Generation with Parametric Motion Transformer.Preprint
Li, B., Zhao, Y., & Sheng, L. (2021) · arXiv (Cornell University) · Cited by 11 · see citing works
Pose Representations for Deep Skeletal Animation
Andreou, N., Aristidou, A., & Chrysanthou, Y. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14632 · Cited by 10 · see citing works
Human motion prediction with gated recurrent unit model of multi-dimensional input
Yu, Y., Tian, N., Hao, X., Ma, T., & Yang, C. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02764-x · Cited by 10 · see citing works
“In this experiment, we also followed the benchmark in [6, 8, 15, 21, 26] and used Subjects 1, 6, 7, 8, 9, and 11 as training data and Subject 5 as test data.”
BackgroundMethodHUMOS: Human Motion Model Conditioned on Body Shape
Tripathi, S., Taheri, O., Lassner, C., Black, M. J., Holden, D., & Stoll, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_8 · Cited by 9 · see citing works
Learning to predict diverse human motions from a single image via mixture density networks
Gu, C., Zhao, Y. F., & Zhang, C. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109549 · Cited by 9 · see citing works
“The majority of previous human motion prediction works [11, 17, 6, 27, 13, 1, 36, 22, 26, 25, 24] rely on the past motion sequence.”
Overlooked Poses Actually Make Sense: Distilling Privileged Knowledge for Human Motion Prediction
Sun, X., Cui, Q., Sun, H., Li, B., Li, W., & Lu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_39 · Cited by 9 · see citing works
3D skeleton-based human motion prediction using spatial–temporal graph convolutional network
Huang, J., & Kang, H. (2024) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-024-00341-9 · Cited by 9 · see citing works
Plan, Posture and Go: Towards Open-Vocabulary Text-to-Motion Generation
Liu, J., Dai, W., Wang, C., Cheng, Y., Tang, Y., & Tong, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73383-3_26 · Cited by 9 · see citing works
“Early efforts [5,12,22,24,25,37,52,60] employed RNNs to model human motion sequence, motivated by the powerful capability in capturing temporal dynamics.”
Chen, S., Peng, C., Yang, B., Lin, J., Zhou, L., Jiang, Z., Liu, Z., Liu, Y., & Tang, L. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00083-5 · Cited by 9 · see citing works
Diverse Motion In-Betweening From Sparse Keyframes With Dual Posture Stitching
Ren, T., Yu, J., Guo, S., et al. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3363457 · Cited by 8 · see citing works
“Naturalness - Normalized Power Spectrum Similarity (NPSS [9]): The similarity of the distribution of the generated motion and the ground truth.”
Geometric algebra-based multiscale encoder-decoder networks for 3D motion prediction
Zhong, J., & Cao, W. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04908-7 · Cited by 8 · see citing works
Ma, J., Zhang, Y., Zhou, H., Yang, H., & Wu, X. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112126 · Cited by 8 · see citing works
Forecasting of 3D Whole-Body Human Poses with Grasping Objects
Yan, H., Cui, Q., Xie, J., & Guo, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00170 · Cited by 8 · see citing works
“Forecasting upcoming 3D human poses, conditioned on the historical ones, is an essential task in human-robot interaction (HRI) [5,6,21,22,25,30,33,34,39,50,62,65].”
BackgroundCOIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation
Li, J., Yuan, Y., Rempe, D., Zhang, H., Molchanov, P. A., Lu, C., Kautz, J., & Iqbal, U. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_24 · Cited by 8 · see citing works
Spatial–temporal modeling for prediction of stylized human motion
Zhong, C., Hu, L., & Xia, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.08.075 · Cited by 8 · see citing works
Human Motion Prediction via Continual Prior Compensation
Tang, J., Hu, J.-F., Liang, T., Lin, X., Sun, J., Zheng, W.-S., & Lai, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3651530 · Cited by 8 · see citing works
“…addressed this task by autoregressively forecasting using recurrent neural networks (RNNs) and trans-former architectures [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], or parallelly generating all frames with graph convolution networks (GCNs) [12], [13], [14], [15], [16], [17],…”
Test-time Personalizable Forecasting of 3D Human Poses
Cui, Q.-J., Sun, H., Lu, J.-Z., Li, W.-Q., Li, B., Yi, H.-W., & Wang, H. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.00032 · Cited by 11 · see citing works
“Moreover, our prediction skeleton (blue) is closer to the ground truth (red) against the vanilla baseline [26]. ploration of this emerging field [16, 5, 2].”
BackgroundFew-shot human motion prediction using deformable spatio-temporal CNN with parameter generation
Zang, C., Li, M., & Pei, M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.130 · Cited by 7 · see citing works
MTAN: Multi-degree Tail-aware Attention Network for human motion prediction
Tang, J., Chen, J., Su, Y. R., Xing, M., & Zhu, S. (2024) · Internet of Things · doi:10.1016/j.iot.2024.101134 · Cited by 7 · see citing works
- Authors
- Gopalakrishnan, A., Mali, A., Kifer, D., Giles, L., & Ororbia, A. G.
- Affiliations
- Pennsylvania State University (US)
- Pennsylva State Univ.
- Pennsylvania State
- Rochester Institute of Techonology
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 12108-12117
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2892040300
- Semantic Scholar
- 057444e01041ffeb50705e0b056532c2e5e15a2e
- Cited by
- 185
- References
- 34
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 43 · 2023: 35 · 2022: 32 · 2021: 27 · 2020: 18 · 2019: 6 · 2018: 1
- FWCI
- 32.10
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (1317560)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- human motion prediction, motion synthesis, multi-step forecasting, closed-loop prediction, multi-objective loss function, user study, joint angle prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
