The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Generating Diverse and Natural 3D Human Motions from Text
Abstract
Automated generation of 3D human motions from text is a challenging problem. The generated motions are expected to be sufficiently diverse to explore the text-grounded motion space, and more importantly, accurately depicting the content in prescribed text descriptions. Here we tackle this problem with a two-stage approach: text2length sampling and text2motion generation. Text2length involves sampling from the learned distribution function of motion lengths conditioned on the input text. This is followed by our text2motion module using temporal variational autoen-coder to synthesize a diverse set of human motions of the sampled lengths. Instead of directly engaging with pose sequences, we propose motion snippet code as our internal motion representation, which captures local semantic motion contexts and is empirically shown to facilitate the generation of plausible motions faithful to the input text. Moreover, a large-scale dataset of scripted 3D Human motions, HumanML3D, is constructed, consisting of 14,616 motion clips and 44,970 text descriptions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“To address the variable length sequence-to-sequence generation task, our training process utilizes both curriculum learning [3] and scheduled sampling [2] strategies, as follows.”
MethodTemporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734 · cited 2×
“Motions from other comparison methods are too distorted to be rendered with the SMPL human shapes [22].”
MethodHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1×
“6M [15], MoVi [9] and BABEL [33], in the form of everyday actions and sports movements.”
BackgroundScheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621
Pixel Recurrent Neural Networks
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06759 · Cited by 1,437
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
AMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323
On the Continuity of Rotation Representations in Neural Networks
Zhou, Y., Barnes, C., Lu, J., Yang, J., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00589 · Cited by 1,255
MoCoGAN: Decomposing Motion and Content for Video Generation
Tulyakov, S., Liu, M.-Y., Yang, X., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00165 · Cited by 1,076
Phase-functioned neural networks for character control
Holden, D., Komura, T., & Saito, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073663 · Cited by 546
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection
Piao, Y., Ji, W., Li, J., Zhang, M., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00735 · Cited by 497
Self-citationAction-Conditioned 3D Human Motion Synthesis with Transformer VAE
Petrovich, M., Black, M. J., & Varol, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01080 · Cited by 493
Action2Motion: Conditioned Generation of 3D Human Motions
Guo, C., Zuo, X., Wang, S., Zou, S., Sun, Q., Deng, A., Gong, M., & Cheng, L. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413635 · Cited by 398
Self-citationKojima, A., Tamura, T., & Fukunaga, K. (2002) · International Journal of Computer Vision · doi:10.1023/a:1020346032608 · Cited by 338
Stochastic Video Generation with a Learned Prior
Denton, E., & Fergus, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07687 · Cited by 278
The KIT Motion-Language Dataset
Plappert, M., Mandery, C., & Asfour, T. (2016) · Big Data · doi:10.1089/big.2016.0028 · Cited by 272
Language2Pose: Natural Language Grounded Pose Forecasting
Ahuja, C., & Morency, L.-P. (2019) · 2019 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00084 · Cited by 271
Memory-Attended Recurrent Network for Video Captioning
Pei, W., Zhang, J., Wang, X., Ke, L., Shen, X., & Tai, Y.-W. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00854 · Cited by 240
BABEL: Bodies, Action and Behavior with English Labels
Punnakkal, A. R., Chandrasekaran, A., Athanasiou, N., Quiros-Ramirez, A., & Black, M. J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00078 · Cited by 186
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning
Lei, J., Wang, L., Shen, Y., Yu, D., Berg, T., & Bansal, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.233 · Cited by 179
Shlizerman, E., Dery, L., Schoen, H., & Kemelmacher-Shlizerman, I. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00790 · Cited by 176
Synthesis of Compositional Animations from Textual Descriptions
Ghosh, A., Cheema, N., Oguz, C., Theobalt, C., & Slusallek, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00143 · Cited by 169
Plappert, M., Mandery, C., & Asfour, T. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.07.006 · Cited by 168
Li, Y., Min, M., Shen, D., Carlson, D., & Carin, L. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12233 · Cited by 163
Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents
Bhattacharya, U., Rewkowski, N., Banerjee, A., Guhan, P., Bera, A., & Manocha, D. (2021) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr50410.2021.00037 · Cited by 150
Reinforced Video Captioning with Entailment Rewards
Pasunuru, R., & Bansal, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1103 · Cited by 131
Pan, Y., Qiu, Z., Yao, T., Li, H., & Mei, T. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3127905 · Cited by 128
Dance Revolution: Long-Term Dance Generation with Music via Curriculum LearningPreprint
Huang, R., Huang, H., Wu, W., Sawada, K. B., Zhang, M., & Jiang, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06119 · Cited by 128
Adversarial Inference for Multi-Sentence Video Description
Park, J. S., Rohrbach, M., Darrell, T., & Rohrbach, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00676 · Cited by 94
A Good Image Generator Is What You Need for High-Resolution Video Synthesis
Tian, Y., Ren, J., Chai, M., Olszewski, K., Peng, X., Metaxas, D. N., & Tulyakov, S. (2021) · arXiv · doi:10.48550/arxiv.2104.15069 · Cited by 88
Memory-oriented Decoder for Light Field Salient Object Detection
Zhang, M., Li, J., Ji, W., Piao, Y., & Lu, H. (2019) · neural information processing systems · Cited by 84
Self-citationEnd-to-End Video Captioning With Multitask Reinforcement Learning
Li, L., & Gong, B. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00042 · Cited by 72
Attentive Semantic Video Generation Using Captions
Marwah, T., Mittal, G., & Balasubramanian, V. N. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.159 · Cited by 69
Yamada, T., Matsunaga, H., & Ogata, T. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2852838 · Cited by 59
Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions
Wang, Z., Yu, P., Zhao, Y., Zhang, R., Zhou, Y., Yuan, J., & Chen, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6911 · Cited by 58
Generating Animated Videos of Human Activities from Natural Language Descriptions
Lin, A. S., Wu, L., Corona, R., Tai, K., Qi-xing Huang, A., & Mooney, R. J. (2018) · Cited by 47
Structure-Aware Human-Action Generation
Yu, P., Zhao, Y., Li, C., Yuan, J., & Chen, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_2 · Cited by 47
Scripted Video Generation With a Bottom-Up Generative Adversarial Network
Chen, Q., Wu, Q., Chen, J., Wu, Q., van den Hengel, A., & Tan, M. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3003227 · Cited by 40
Takeuchi, K., Hasegawa, D., Shirakawa, S., Kaneko, N., Sakuta, H., & Sumi, K. (2017) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3125739.3132594 · Cited by 38
Point-to-Point Video Generation
Wang, T.-H. J., Cheng, Y.-C., Lin, C. H., Chen, H.-T., & Sun, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01059 · Cited by 30
Action2video: Generating Videos of Human 3D Actions
Guo, C., Zuo, X., Wang, S., Liu, X., Zou, S., Gong, M., & Cheng, L. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01550-z · Cited by 22
Self-citation
Cited by
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P.-A., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 538
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437
MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355414 · Cited by 424
Executing your Commands via Motion Diffusion in Latent Space
Chen, X., Jiang, B., Liu, W., Huang, Z., Fu, B., Chen, T., & Yu, G. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01726 · Cited by 356
MotionCLIP: Exposing Human Motion Generation to CLIP Space
Tevet, G., Gordon, B., Hertz, A., Bermano, A. H., & Cohen-Or, D. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20047-2_21 · Cited by 317
Metadata
- Authors
- Guo, C., Zou, S., Zuo, X., Wang, S., Ji, W., Li, X., & Cheng, L.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 5142-5151
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312635677
- Cited by
- 575
- References
- 42
- FWCI
- 176.29
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.