The DOI matches its DataCite record; 82 of the 85 references it cites were found in a registry or an index.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Open accessAbstract
Human motion modeling is important for many modern graphics applications, which typically require professional skills. In order to remove the skill barriers for laymen, recent motion generation methods can directly generate human motions conditioned on natural languages. However, it remains challenging to achieve diverse and fine-grained motion generation with various text inputs. To address this problem, we propose MotionDiffuse, the first diffusion model-based text-driven motion generation framework, which demonstrates several desired properties over existing methods. 1) Probabilistic Mapping. Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected. 2) Realistic Synthesis. MotionDiffuse excels at modeling complicated data distribution and generating vivid motion sequences. 3) Multi-Level Manipulation. MotionDiffuse responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts. Our experiments show MotionDiffuse outperforms existing SoTA methods by convincing margins on text-driven motion generation and action-conditioned motion generation. A qualitative analysis further demonstrates MotionDiffuse's controllability for comprehensive motion generation. Homepage: https://mingyuan-zhang.github.io/projects/MotionDiffuse.html
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Systems exist that directly generate physical representations from texts (T2M-GPT [216], MotionGPT [96], MotionDiffuse [217], and ReMoDiffuse [218]).”
From Past, Present, and Future of Sensor-based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task (2025) · cites this paper as [217]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Research works printed but not linked: 9
- Futrelle R, Speckert G (1978) Extraction of motion data by interactive processing. In: Proc. 1978 IEEE Conf. Pattern Recog. and Image Processing, Chicago, IL, pp 405-408
- Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde- Farley D, Ozair S, Courville A, Bengio Y (2014) Gen- erative adversarial nets. Advances in neural informa- tion processing systems 27
- Henter GE, Alexanderson S, Beskow J (2020) Moglow: Probabilistic and controllable motion synthesis us- ing normalising flows. ACM Transactions on Graph- ics (TOG) 39(6):1-14
- Hong F, Zhang M, Pan L, Cai Z, Yang L, Liu Z (2022) Avatarclip: Zero-shot text-driven generation and ani- mation of 3d avatars. arXiv preprint arXiv:220508535
- Kingma DP, Dhariwal P (2018) Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems 31
- Kingma DP, Welling M (2013) Auto-encoding varia- tional bayes. arXiv preprint arXiv:13126114
- Mukai T, Kuriyama S (2005) Geostatistical motion in- terpolation. In: ACM SIGGRAPH 2005 Papers, pp 1062-1070
- Petrovich M, Black MJ, Varol G (2022) Temos: Gen- erating diverse human motions from textual descrip- tions. arXiv preprint arXiv:220414109
- Tulyakov S, Liu MY, Yang X, Kautz J (2018) Moco- gan: Decomposing motion and content for video gen- eration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1526- 1535
Cited by
Sorted by citations
DiffusionDet: Diffusion Model for Object Detection
Chen, S., Sun, P., Song, Y., & Luo, P. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01816 · Cited by 657
Executing your Commands via Motion Diffusion in Latent Space
Chen, X., Jiang, B., Liu, W., Huang, Z., Fu, B., Chen, T., & Yu, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01726 · Cited by 452
Generating Human Motion from Textual Descriptions with Discrete Representations
Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., & Ying, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01415 · Cited by 328
“Note that MDM [66] and MotionDiffuse [74] evaluate their models with the ground-truth motion length, which is not practical for real applications.”
BackgroundMethodResultInfluentialEDGE: Editable Dance Generation From Music
Tseng, J., Castellon, R., & Liu, C. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00051 · Cited by 332
PhysDiff: Physics-Guided Human Motion Diffusion Model
Yuan, Y., Song, J., Iqbal, U., Vahdat, A., & Kautz, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01467 · Cited by 311
MoMask: Generative Masked Modeling of 3D Human Motions
Guo, C., Mu, Y., Javed, M. G., Wang, S., & Cheng, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00186 · Cited by 280
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Huang, S., Wang, Z., Li, P., Jia, B., Liu, T., Zhu, Y., Liang, W., & Zhu, S.-C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01607 · Cited by 192
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents
Ao, T., Zhang, Z., & Liu, L. (2023) · ACM Transactions on Graphics · doi:10.1145/3592097 · Cited by 175
MoFusion: A Framework for Denoising-Diffusion-Based Motion Synthesis
Dabral, R., Mughal, M. H., Golyanik, V., & Theobalt, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00941 · Cited by 202
MotionDiffuser: Controllable Multi-Agent Motion Prediction Using Diffusion
Jiang, C. M., Cornman, A., Park, C., Sapp, B., Zhou, Y., & Anguelov, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00930 · Cited by 208
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
Zhang, M., Guo, X., Pan, L., Cai, Z., Hong, F., Li, H., Yang, L., & Liu, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00040 · Cited by 238
Self-citationDiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation
Shen, S., Zhao, W., Meng, Z., Li, W., Zhu, Z., Zhou, J., & Lu, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00197 · Cited by 133
InterGen: Diffusion-Based Multi-human Motion Generation Under Complex Interactions
Liang, H., Zhang, W., Li, W., Jingyi Yu, A., & Xu, L. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02042-6 · Cited by 172
DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models
Yang, S., Wu, Z., Li, M., Zhang, Z., Lei, H., Bao, W., Cheng, M., & Xiao, L. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/650 · Cited by 150
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00870 · Cited by 139
InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion
Xu, S., Li, Z., Wang, Y.-X., & Gui, L.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01371 · Cited by 126
Expressive 3D Facial Animation Generation Based on Local-to-Global Latent Diffusion
Song, W., Wang, X., Jiang, Y., Li, S., Hao, A., Hou, X., & Qin, H. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456213 · Cited by 110
HumanMAC: Masked Motion Completion for Human Motion Prediction
Chen, L.-H., Zhang, J., Li, Y., Pang, Y., Xia, X., & Liu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00875 · Cited by 119
Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model
Du, Y., Kips, R., Pumarola, A., Starke, S., Thabet, A., & Sanakoyeu, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00054 · Cited by 126
ZeroEGGS: Zero‐shot Example‐based Gesture Generation from Speech
Ghorbani, S., Ferstl, Y., Holden, D., Troje, N. F., & Carbonneau, M. (2023) · Computer Graphics Forum · doi:10.1111/cgf.14734 · Cited by 115
Ego-Body Pose Estimation via Ego-Head Pose Estimation
Li, J., Liu, C. K., & Wu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01644 · Cited by 106
MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Zhang, Y., Huang, D., Liu, B., Tang, S., Lu, Y., Chen, L., Bai, L., Chu, Q., Yu, N., & Ouyang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28567 · Cited by 128
Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning
Feng, C.-M., Yu, K., Liu, Y., Khan, S. H., & Zuo, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00255 · Cited by 116
FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation
Li, R., Zhao, J., Zhang, Y., Su, M., Ren, Z., Zhang, H., Tang, Y., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00939 · Cited by 95
Object Motion Guided Human Motion Synthesis
Li, J., Wu, J., & Liu, C. K. (2023) · ACM Transactions on Graphics · doi:10.1145/3618333 · Cited by 110
MMM: Generative Masked Motion Model
Pinyoanuntapong, E., Wang, P., Lee, M., & Chen, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00153 · Cited by 114
AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism
Zhong, C., Hu, L., Zhang, Z., & Xia, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00053 · Cited by 89
Motion Mamba: Efficient and Long Sequence Motion Generation
Zhang, Z., Liu, A., Reid, I. D., Hartley, R. I., Zhuang, B., & Tang, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_15 · Cited by 108
ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation
Xu, L., Song, Z., Wang, D., Su, J., Fang, Z., Ding, C., Gan, W., Yan, Y., Jin, X., Yang, X., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00212 · Cited by 67
NetDiffus: Network traffic generation by diffusion models through time-series imaging
Sivaroopan, N., Bandara, D., Madarasingha, C., Jourjon, G., Jayasumana, A. P., & Thilakarathna, K. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.110616 · Cited by 75
Li, R., Zhang, Y.-X., Zhang, Y., Zhang, H., Guo, J., Zhang, Y. T., Liu, Y., & Li, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00151 · Cited by 77
FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Models
Xu, J., Guo, Y., & Peng, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00060 · Cited by 70
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3678545 · Cited by 61
MotionLCM: Real-Time Controllable Motion Generation via Latent Consistency Model
Dai, W., Chen, L.-H., Wang, J., Liu, J., Dai, B., & Tang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_22 · Cited by 61
UDE: A Unified Driving Engine for Human Motion Generation
Zhou, Z., & Wang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00545 · Cited by 69
Chen, J., Liu, Y., Wang, J., Zeng, A., Li, Y., & Chen, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00702 · Cited by 70
CIRCLE: Capture In Rich Contextual Environments
Araújo, J. P., Li, J., Vetrivel, K., Agarwal, R., Wu, J., Gopinath, D., Clegg, A., & Liu, C. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02032 · Cited by 66
SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation
Athanasiou, N., Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00916 · Cited by 62
Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation
Azadi, S., Shah, A., Hayes, T., Parikh, D., & Gupta, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01381 · Cited by 54
Flexible Motion In-betweening with Diffusion Models
Cohan, S., Tevet, G., Reda, D., Peng, X. B., & van de Panne, M. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657414 · Cited by 49
Priority-Centric Human Motion Generation in Discrete Latent Space
Kong, H., Gong, K., Lian, D., Mi, M. B., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01360 · Cited by 56
Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model
Wang, Y., Leng, Z., Li, F. W. B., Wu, S., & Liang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02014 · Cited by 56
Move as you Say, Interact as you can: Language-Guided Human Motion Generation with Scene Affordance
Wang, Z., Chen, Y., Jia, B., Li, P., Zhang, J., Zhang, J., Liu, T., Zhu, Y., Liang, W., & Huang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00049 · Cited by 67
DiffusionTrack: Point Set Diffusion Model for Visual Object Tracking
Xie, F., Wang, Z., & Ma, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01808 · Cited by 58
EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation
Zhou, W., Dou, Z., Cao, Z., Liao, Z., Wang, J., Wang, W., Liu, Y., Komura, T., Wang, W., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72627-9_2 · Cited by 71
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations
Yao, H., Song, Z., Zhou, Y., Ao, T., Chen, B., & Liu, L. (2024) · ACM Transactions on Graphics · doi:10.1145/3658137 · Cited by 46
BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance
Qin, H., Ma, X., Ding, Y., Li, X., Zhang, Y., Ma, Z., Wang, J., Luo, J., & Liu, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3243259 · Cited by 45
TLControl: Trajectory and Language Control for Human Motion Synthesis
Wan, W., Dou, Z., Komura, T., Wang, W., Jayaraman, D., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_3 · Cited by 45
NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis
Kulkarni, N. V., Rempe, D., Genova, K., Kundu, A., Johnson, J., Fouhey, D. F., & Guibas, L. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00096 · Cited by 68
Controllable Human-Object Interaction Synthesis
Li, J., Clegg, A., Mottaghi, R., Wu, J., Puig, X., & Liu, C. K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72940-9_4 · Cited by 84
Speech driven video editing via an audio-conditioned diffusion model
Bigioi, D., Basak, S., Stypułkowski, M., Zięba, M., Jordan, H., McDonnell, R., & Corcoran, P. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.104911 · Cited by 49
Text2HOI: Text-Guided 3D Motion Generation for Hand-Object Interaction
Cha, J., Kim, J., Yoon, J. S., & Baek, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00156 · Cited by 55
Optimizing Diffusion Noise Can Serve As Universal Motion Priors
Karunratanakul, K., Preechakul, K., Aksan, E., Beeler, T., Suwajanakorn, S., & Tang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00133 · Cited by 69
DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion
Nag, S., Zhu, X., Deng, J., Song, Y.-Z., & Xiang, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00951 · Cited by 47
HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models
Peng, X., Xie, Y., Wu, Z., Jampani, V., Sun, D., & Jiang, H. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00271 · Cited by 40
Leng, Z., Kwon, H., & Ploetz, T. (2023) · Proceedings of the 2023 International Symposium on Wearable Computers · doi:10.1145/3594738.3611361 · Cited by 39
Being Comes from Not-Being: Open-Vocabulary Text-to-Motion Generation with Wordless Training
Lin, J., Chang, J., Liu, L., Li, G., Lin, L., Tian, Q. C., & Chen, C. W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02224 · Cited by 43
SMooDi: Stylized Motion Diffusion Model
Zhong, L., Xie, Y., Jampani, V., Sun, D., & Jiang, H. W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_23 · Cited by 38
Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation
Wang, Y., Mu, L., Liu, J., Leng, Z., Li, F. W. B., Zhang, Z., & Liang, X. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02392-9 · Cited by 44
Role-aware Interaction Generation from Textual Description
Tanaka, M., & Fujiwara, K. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01466 · Cited by 50
LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation
Zhi, Y., Cun, X., Chen, X., Shen, X., Guo, W. Q., Huang, S., & Gao, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01902 · Cited by 44
PACE: Human and Camera Motion Estimation from in-the-wild Videos
Kocabas, M., Yuan, Y., Molchanov, P. A., Guo, Y., Black, M. J., Hilliges, O., Kautz, J., & Iqbal, U. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00103 · Cited by 41
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729467 · Cited by 33
Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models
Pi, H., Peng, S., Yang, M., Zhou, X., & Bao, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01383 · Cited by 37
Seamless Human Motion Composition with Blended Positional Encodings
Barquero, G., Escalera, S., & Palmero, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00051 · Cited by 44
REMOS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions
Ghosh, A., Dabral, R., Golyanik, V., Theobalt, C., & Slusallek, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72764-1_24 · Cited by 33
Wu, D., Zheng, P., Zhao, Q., Zhang, S., Qi, J., Hu, J., Zhu, G., & Wang, L. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103064 · Cited by 38
Bailando ++: 3D Dance GPT With Choreographic Memory
Li, S., Yu, W., Gu, T., Lin, C., Wang, Q., Qian, C., Loy, C. C., & Liu, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3319435 · Cited by 36
Self-citationTaming Diffusion Probabilistic Models for Character Control
Chen, R., Shi, M., Huang, S., Tan, P., Komura, T., & Chen, X. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657440 · Cited by 32
Wouwe, T. V., Lee, S., Falisse, A. M. D., Delp, S. L., & Liu, C. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00243 · Cited by 43
CoMo: Controllable Motion Generation Through Language Guided Pose Code Editing
Huang, Y., Wan, W., Yang, Y. S., Callison-Burch, C., Yatskar, M., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_11 · Cited by 32
Digital Life Project: Autonomous 3D Characters with Social Intelligence
Cai, Z., Jiang, J., Qing, Z., Guo, X., Zhang, M., Lin, Z., Mei, H., Chen, W., Wang, R., Yin, W., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00062 · Cited by 42
Self-citationFineStyle: Semantic-Aware Fine-Grained Motion Style Transfer with Dual Interactive-Flow Fusion
Song, W., Jin, X., Li, S., Chen, C., Hao, A., & Hou, X. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3320216 · Cited by 30
DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
Qi, Q., Zhuo, L., Zhang, A., Liao, Y., Fang, F., Liu, S., & Yan, S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612307 · Cited by 38
BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis
Castillo, A., Escobar, M., Jeanneret, G., Pumarola, A., Arbeláez, P., Thabet, A., & Sanakoyeu, A. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00456 · Cited by 42
Dance with You: The Diversity Controllable Dancer Generation via Diffusion Models
Yao, S., Sun, M., Li, B., Yang, F., Wang, J., & Zhang, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612046 · Cited by 31
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 32
StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Framework
Huang, Y., Yang, H., Luo, C., Wang, Y., Xu, S., Zhang, Z., Zhang, M., & Peng, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681657 · Cited by 42
Self-citationFLAG3D: A 3D Fitness Activity Dataset with Language Instruction
Tang, Y., Liu, J., Liu, A., Yang, B., Dai, W., Rao, Y., Lu, J., Zhou, J., & Li, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02117 · Cited by 32
The DiffuseStyleGesture+ entry to the GENEA Challenge 2023
Yang, S., Xue, H., Zhang, Z., Li, M., Wu, Z., Wu, X., Xu, S., & Dai, Z. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3577190.3616114 · Cited by 33
Social Diffusion: Long-term Multiple Human Motion Anticipation
Tanke, J., Zhang, L., Zhao, A. T., Tang, C., Cai, Y., Wang, L., Wu, P.-C., Gall, J., & Keskin, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00880 · Cited by 30
OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers
Liang, H., Bao, J., Zhang, R., Ren, S., Xu, Y., Yang, S., Chen, X., Jingyi Yu, A., & Xu, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00053 · Cited by 33
Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generation
Chen, B., Li, Y., Ding, Y.-X., Shao, T., & Zhou, K. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680847 · Cited by 38
Iterative Motion Editing with Natural Language
Goel, P., Wang, K.-C. J., Liu, C. K., & Fatahalian, K. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657447 · Cited by 35
Towards Variable and Coordinated Holistic Co-Speech Motion Generation
Liu, Y., Cao, Q., Wen, Y., Jiang, H., & Ding, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00155 · Cited by 39
BeautyDiffusion: Generative latent decomposition for makeup transfer via diffusion models
Feng, H., Li, H., Ning, X., & Li, Q. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103241 · Cited by 28
Can We Use Diffusion Probabilistic Models for 3D Motion Prediction?
Ahn, H., Mascaro, E. V., & Lee, D. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160722 · Cited by 29
Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views
Zhang, S., Ma, Q., Zhang, Y., Aliakbarian, S., Cosker, D. P., & Tang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00734 · Cited by 28
FreeMotion: A Unified Framework for Number-Free Text-to-Motion Synthesis
Fan, K., Tang, J., Cao, W., Yi, R., Li, M., Gong, J., Zhang, J., Wang, Y., Wang, C., & Ma, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_6 · Cited by 25
HanDiffuser: Text-to-Image Generation with Realistic Hand Appearances
Narasimhaswamy, S., Bhattacharya, U., Chen, X., Dasgupta, I., Mitra, S., & Hoai, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00239 · Cited by 38
ParCo: Part-Coordinating Text-to-Motion Synthesis
Zou, Q., Yuan, S., Du, S., Wang, Y., Liu, C., Xu, Y., Chen, J., & Ji, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72992-8_8 · Cited by 42
4D Facial Expression Diffusion Model
Zou, K., Faisan, S., Yu, B., Valette, S., & Seo, H. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3653455 · Cited by 28
TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis
Zhang, Z., Liu, R., Hanocka, R., & Aberman, K. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657515 · Cited by 39
MagicTime: Time-Lapse Video Generation Models as Metamorphic Simulators
Yuan, S., Huang, J., Shi, Y., Xu, Y., Zhu, R., Lin, B., Cheng, X., Yuan, L., & Luo, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3558507 · Cited by 50
POPDG: Popular 3D Dance Generation with PopDanceSet
Luo, Z., Ren, M., Hu, X., Huang, Y., & Li, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02548 · Cited by 26
Large Motion Model for Unified Multi-modal Motion Generation
Zhang, M., Jin, D., Gu, C., Hong, F., Cai, Z., Huang, J., Zhang, C., Guo, X., Yang, L., He, Y., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_23 · Cited by 49
Self-citationDiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions
Christen, S., Hampali, S., Şener, F., Remelli, E., Hodaň, T., Sauser, E. L., Ma, S., & Tekin, B. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687563 · Cited by 44
Xiao, L., Lu, S., Pi, H., Fan, K., Pan, L., Zhou, Y., Feng, Z., Zhou, X., Peng, S., & Wang, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00940 · Cited by 19
Self-citationSATO: Stable Text-to-Motion Framework
Chen, W., Xiao, H., Zhang, E., Hu, L., Wang, L., Liu, M., & Chen, C. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681034 · Cited by 19
MAS: Multi-view Ancestral Sampling for 3D Motion Generation Using 2D Diffusion
Kapon, R., Tevet, G., Cohen-Or, D., & Bermano, A. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00192 · Cited by 22
Metadata
- Authors
- Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z.
- Affiliations
- Nanyang Technological University (SG)
- Shanghai Artificial Intelligence Laboratory (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Volume
- 46
- Issue
- 6
- Pages
- 4115-4128
- ISSN
- 0162-8828 (ISSN-L)
- arXiv
- 2208.15001
- OpenAlex
- W4391305822
- Cited by
- 437
- References
- 85
- Citations by year
- 2026: 118 · 2025: 148 · 2024: 118 · 2023: 53
- FWCI
- 60.01
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Ministry of Education, India
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- human motion modeling, diffusion models, probabilistic mapping, motion synthesis
- MeSH terms
- Algorithms, Computer Graphics, Humans, Image Processing, Computer-Assisted, Motion, Movement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
