The DOI matches its Crossref record; 90 of the 94 references it cites were found in a registry or an index.
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
Open accessAbstract
We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music. The proposed AIST++ dataset contains 5.2 hours of 3D dance motion in 1408 sequences, covering 10 dance genres with multi-view videos with known camera poses—the largest dataset of this kind to our knowledge. We show that naively applying sequence models such as transformers to this dataset for the task of music conditioned 3D motion generation does not produce satisfactory 3D motion that is well correlated with the input music. We overcome these shortcomings by introducing key changes in its architecture design and supervision: FACT model involves a deep cross-modal transformer block with full-attention that is trained to predict N future motions. We empirically show that these changes are key factors in generating long sequences of realistic dance motion that are well-attuned to the input music. We conduct extensive experiments on AIST++ with user studies, where our method outperforms recent state-of-the-art methods both qualitatively and quantitatively. The code and the dataset can be found at: https://google.github.io/aichoreographer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We note that the simple prediction-based late fusion [247], [248] is less adopted in MML Transformers.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [248]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 8×
“We propose Full Attention Cross-Modal Transformer (FACT) model for the task of 3D dance motion generation.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“All our experiments are trained with 16 batch size using Adam [45] optimizer.”
MethodJakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 1×
“Training data for 2D pose and audio is abundant thanks to the high reliability of 2D pose detectors [14].”
BackgroundDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 2×
“While there are many large scale 3D motion capture datasets [39, 59, 1, 37], mocap dataset of 3D dance is quite limited as it requires heavy instrumentation and expert dancers for capture.”
BackgroundMethod
Research works printed but not linked: 3
- lamothe, 2019, The dancing species: how moving together in time helps make us human, Aeon
- alemi, 2017, Groovenet: Real-time music-driven dance movement generation using artificial neural networks, Networks, 8, 26
- chen, 2021, Choreomaster: choreography-oriented music-driven dance synthesis, ACM Transactions on Graphics (TOG), 40, 1
Cited by
Sorted by citations
Items 201–300 of 553
Dance Information Processing: Computational Approaches for Assisting Dance Composition
Tsuchida, S. (2024) · New Generation Computing · doi:10.1007/s00354-024-00273-2 · Cited by 8
“This database was subsequently augmented to include a 3D motion dataset [2].”
MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation
Mao, X., Jiang, Z., Wang, Q., Fu, C., Zhang, J., Wu, J., Wang, Y., Wang, C., Li, W., & Chi, M. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680684 · Cited by 12
Doodle Your Motion: Sketch-Guided Human Motion Generation
Wu, Z., Wang, Q., Zheng, X., Ye, J., Yang, P., Wang, Y., & Wang, Y. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3521333 · Cited by 9
“Various conditions have been explored such as past motions [22], [23], [25], [37], [41], [54], music [3], [33], [50], action labels [21], [42], scene [10], [24], [53], text [20], [31], [34], [43].”
Dance any Beat: Blending Beats with Visuals in Dance Video Generation
Wang, X., Wang, H., Liu, D., & Cai, W. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00502 · Cited by 8
“The AIST++ dataset [22], a rich compilation of 3D motion data, has significantly advanced the field.”
InfluentialAI-Powered Choreography Using a Multilayer Perceptron Model for Music-Driven Dance Generation
Zeng, D. (2025) · Informatica · doi:10.31449/inf.v49i20.8103 · Cited by 8
MaskControl: Spatio-Temporal Control for Masked Motion Synthesis
Pinyoanuntapong, E., Saleem, M. U., Karunratanakul, K., Wang, P., Xue, H., Chen, C., Guo, C., Cao, J., Ren, J., & Tulyakov, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00928 · Cited by 17
“Example control modalities include music [26, 27, 29, 44, 45, 49], interacting object [2, 9, 25, 28], tracking sensors [11], scene [17, 52] programmable motion [30], style [62], goal-reaching task [10], and multi-track timeline control [38].”
Action-Conditioned Generation of Bimanual Object Manipulation Sequences
Razali, H., & Demiris, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25308 · Cited by 8
“These models are then trained to generate the complete motion conditioned either on an audio signal (Li et al. 2021), a semantic action label (Guo et al.”
Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control
Xu, Z., Zhang, Y., Yang, S., Li, R., & Li, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28458 · Cited by 10
Self-citationZhang, X., Zhu, S., Zhao, Y., Jia, M., & Zhu, Q. (2024) · Information Research · doi:10.47989/ir292856 · Cited by 8
“This expansion is evident in AI's ability to generate new content in creative and artistic fields, including drawing (Sun et al., 2019), music composition (Choi et al., 2016), fiction writing (Yang et al., 2022), urban planning (Cooney, 2021) and dance (Li et al., 2021).”
BackgroundKim, S., & Lee, K. (2024) · IEEE Access · doi:10.1109/access.2024.3420433 · Cited by 8
“To assess how diverse the dance results produced by this model are in terms of motion, we decided to calculate motion diversity using the methodology used in previous studies [37].”
BackgroundMethodInfluentialDIM: Dyadic Interaction Modeling for Social Behavior Generation
Tran, M., Chang, D., Siniukov, M., & Soleymani, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_27 · Cited by 8
A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning
Vilaça, L., Yu, Y., & Viana, P. (2025) · ACM Computing Surveys · doi:10.1145/3696445 · Cited by 17
“We believe that leveraging the generative nature of GANs (i.e., artificial data) will allow the development of more complex self-supervised frameworks, where the proxy tasks depend on samples generated with the learned data distribution.”
Towards Stable Human Pose Estimation via Cross-View Fusion and Foot Stabilization
Zhuo, L., Cao, J., Wang, Q., Zhang, B., & Bo, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00070 · Cited by 8
Exploring Multi-Modal Control in Music-Driven Dance Generation
Li, R., Dai, Y., Zhang, Y., Li, J., Yang, J. J., Guo, J., & Li, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447825 · Cited by 14
“Early methods [3, 4] input music and seed motions into a single network, such as Transformer [5], generating new dance movements frame by frame in an autoregressive manner.”
MethodSelf-citationYang, S., Gu, X., Kuang, Z., Qin, F., & Wu, Z. (2024) · The Visual Computer · doi:10.1007/s00371-024-03641-7 · Cited by 7
“AIST++ dataset [197] is constructed from the AIST Dance Video DB dataset [198].”
Self-citationTIA2V: Video generation conditioned on triple modalities of text–image–audio
Zhao, M., Wang, W., Zhang, R., Jia, H., & Chen, Q. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126278 · Cited by 9
TED-culture: culturally inclusive co-speech gesture generation for embodied social agents
Shen, Y., & Johal, W. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1546765 · Cited by 11
“The Beat Consistency Score (BC) Li et al. (2022), Li et al. (2021) is designed to gauge the correlation between motion and speech beats.”
LS-GAN: Human Motion Synthesis with Latent-Space GANs
Amballa, A., Akkinapalli, G., & Muralikrishnan, V. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00039 · Cited by 8
“Conditional motion synthesis, on the other hand, employs inputs from various modalities such as music [28] and text [24] to generate motion sequences.”
Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis
Tang, H., Shao, L., Zhang, Z., Gool, L. V., & Sebe, N. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3588237 · Cited by 7
“We also provide an inference time comparison between our method, FACT [32], DanceFormer [18], and EDGE [81] in Table 2.”
Chen, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-16345-2 · Cited by 7
“The AIST++ dataset 31 is a large-scale dataset aimed at advancing research in dance motion generation 32 , choreography analysis 33 , and motion-music alignment.”
Implicit Compositional Generative Network for Length-Variable Co-Speech Gesture Synthesis
Xu, C., Yan, J., Yang, Y., & Deng, C. D. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3348331 · Cited by 7
“6) BC: Beat Consistency Score (BC) is wildly used in motion-audio tasks [11], [24], [53], [54], which computes the average distance between every music beat and its nearest kinematic beat.”
MethodZhong, Y., & Demiris, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28893 · Cited by 9
A Geometry Loss Combination for 3D Human Pose Estimation
Matsune, A., Hu, S., Li, G., Wen, S., Zhu, X., & Tan, Z. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00324 · Cited by 7
Speech-Driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference
Zhang, F., Wang, Z., Lyu, X., Zhao, S., Li, M., Geng, W., Ji, N., Du, H., Gao, F., Wu, H., et al. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3393236 · Cited by 9
“2) Objective Evaluation: We introduce three objective evaluation metrics: Fr ´ echet Gesture Distance (FGD) in both feature and raw data spaces [49], and BeatAlign [50].”
SynSP: Synergy of Smoothness and Precision in Pose Sequences Refinement
Wang, T., Jin, L., Wang, Z., Li, J., Li, L., Zhao, F., Cheng, Y., Li, Y., Zhou, L. P., Xing, J., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00179 · Cited by 9
“6M [12], AIST++ [20], 3DPW [35], and CMU-Mocap [33] datasets.”
CustomListener: Text-Guided Responsive Interaction for User-Friendly Listening Head Generation
Liu, X., Guo, Y., Zhen, C., Li, T., Ao, Y., & Yan, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00234 · Cited by 13
“Currently, many applications, such as digital avatar generation [8, 20] and human-computer interaction [45], use the listener agent to simulate more realistic conversations, which has promoted many studies in LHG.”
BackgroundLearning Uncoupled-Modulation CVAE for 3D Action-Conditioned Human Motion Synthesis
Zhong, C., Hu, L., Zhang, Z., & Xia, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_42 · Cited by 7
“In addition, some other condition ways include conditioned on contextual cues and interaction with objects [7], conditioned on music [20], etc. Unlike these works, the problem we want to solve is conditioned on action labels to generate motion sequences without any initial frame or past sequence.”
BackgroundVAST: Vivify Your Talking Avatar via Zero-Shot Expressive Facial Style Transfer
Chen, L., Wu, Z., Li, R., Bao, W., Ling, J., Tan, X., & Zhao, S. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00320 · Cited by 8
“We adopt the diversity metric to estimate the expressiveness of facial movements, which has been widely adopted in the gesture and dance generation field [20, 12, 21, 32].”
Self-citationEfficient 3D Articulated Human Generation with Layered Surface Volumes
Xu, Y., Wang, Y., Bergman, A. W., Chai, M., Zhou, B., & Wetzstein, G. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00047 · Cited by 7
Real-Time Full-body Interaction with AI Dance Models: Responsiveness to Contemporary Dance
Zhou, J., Weber, R., Wen, E., & Lottridge, D. M. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712077 · Cited by 6
“To answer our research question, we adapt various deep-learning models [6, 8, 17], and especially focused on Bailando++ [17], to generate shortbursts ofoutput inresponse toa short inputsequence.”
InfluentialMulti‐Modal Instrument Performances (MMIP): A Musical Database
Kyriakou, T., Aristidou, A., & Charalambous, P. (2025) · Computer Graphics Forum · doi:10.1111/cgf.70025 · Cited by 6
Foundation models for exploration geophysics
Liu, Q., Chen, Z., & Ma, J. (2025) · Geophysics · doi:10.1190/geo2024-0916.1 · Cited by 9
A Survey on Human Interaction Motion Generation
Sui, K., Ghosh, A., Hwang, I., Zhou, B., Wang, J., & Guo, C. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02582-5 · Cited by 22
“1: Statistics on the number of works and datasets on human interaction motion generation over the past two decades, categorized into four interaction scenarios.”
Dai, Y., Zhu, W., Li, R., Li, X. Y., Zhang, Z., Li, J., & Yang, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02611-3 · Cited by 6
“Due to dancer swap phenomena, naive long diffusion sampling (EDGE, GCD) causes severe abrupt swapping in long-duration generation, as the absence of spatial information disrupts position consistency across epochs.”
InfluentialSelf-citationYang, Q., Huang, L., Wang, K., Guan, J., He, S., Li, F., Zhou, H., Yu, L. J., Li, Y., Feng, H., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01172 · Cited by 8
SONIC: Supersizing motion tracking for natural humanoid whole-body control
Luo, Z., Yuan, Y., Wang, T., Li, C., Castañeda, F., Chen, S., Cao, Z.-A., Li, J., Minor, D., Ben, Q., et al. (2026) · Science Robotics · doi:10.1126/scirobotics.aed4592 · Cited by 228
“While there exists prior art in motion tracking (Chen et al., 2025; He et al., 2024, 2025; Liao et al., 2025; Luo et al., 2023; Wang et al., 2020; Yin et al., 2025; Zeng et al., 2025; Zhang et al., 2025), they are mostly limited to showing whole-body motion tracking results on training data and…”
PCMG:3D point cloud human motion generation based on self-attention and transformer
Ma, W., Yin, M., Li, G., Feng, Y., & Chang, K. (2023) · The Visual Computer · doi:10.1007/s00371-023-03063-x · Cited by 7
“Conditioned motion generation tasks involve various types of input data, such as music [6, 25–27], speech audio [28, 29], text [7, 8, 10, 17, 30–32], and action labels [3, 5, 8–10].”
Human from Blur: Human Pose Tracking from Blurry Images
Zhao, Y., Rozumnyi, D., Song, J., Hilliges, O., Pollefeys, M., & Oswald, M. R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01369 · Cited by 7
Human-AI Co-creation for Intangible Cultural Heritage Dance: Cultural Genes Retaining and Innovation
Zhu, H., Zhou, X., & Liu, H. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-61950-2_46 · Cited by 6
DanceMimic: Awaken Your Dancing Instinct through a Real-time Dance Imitation Capture System
Kim, S., Huh, J., Park, Y., Kim, J. S., & Lee, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3684991 · Cited by 6
“We captured three subjects imitating ten choreography sequences sampled from the AIST++ dataset [9].”
Kim, H. S. (2025) · Applied Sciences · doi:10.3390/app15010409 · Cited by 7
Pang, H., Ding, T., He, L., Tao, M., Zhang, L., & Gan, Q. (2025) · International Conference on Computer Graphics and Virtuality (ICCGV) · doi:10.1117/12.3060395 · Cited by 6
“Beat Alignment Score was introduced by 49 to measure the alignment of audio beats and motion beats.”
MethodAdvanced Dance Choreography System Using Bidirectional LSTMs
Yoo, H., & Sung, Y. (2023) · Systems · doi:10.3390/systems11040175 · Cited by 6
PirouNet: Creating Dance Through Artist-Centric Deep Learning
Papillon, M., Pettee, M., & Miolane, N. (2023) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-28993-4_31 · Cited by 11
“Lastly, other methods prompt sequences with music [32,30,1,31], which a choreographer can use to influence dance outputs.”
A Two-Part Transformer Network for Controllable Motion Synthesis
Hou, S., Tao, H., Bao, H., & Xu, W. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3284402 · Cited by 9
“Researchers also used a cross-modal Transformer to synthesize dance motions conditioned on music [9], [18], [49], [50].”
MethodM2C: Concise Music Representation for 3D Dance Generation
Marchellus, M., & Park, I. K. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00337 · Cited by 6
“Recent works have used a learning based method for dance generations such as CNN [27, 10], RNN [44, 49, 4], GAN [48, 1, 41], and transformers network [46, 30, 28, 29, 14] to learn the correlation between musical attributes and corresponding dance moves.”
BackgroundMethodInfluentialMotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation
Hoang, N. M., Gong, K., Guo, C., & Mi, M. B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.27988 · Cited by 11
DanceAnyWay: Synthesizing Beat-Guided 3D Dances with Randomized Temporal Contrastive Learning
Bhattacharya, A., Paranjape, M., Bhattacharya, U., & Bera, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27836 · Cited by 6
“They also marked our generated dances 3 or more in motion quality for 82% of the samples, compared to 59% for FACT, 34% for Bailando, and 73% for EDGE.”
BackgroundMethodInfluentialCascaded Siamese Self-supervised Audio to Video GAN
Aldausari, N., Sowmya, A., Marcus, N., & Mohammadi, G. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00515 · Cited by 6
“One such category is dance retargeting, where the training phase uses key points of the skeleton [4, 15, 19, 27, 33, 38, 39, 53, 57] or 3D body poses [28].”
BackgroundXu, Q., Cao, R., Shen, X., Du, H., Wang, S., & Yu, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01147 · Cited by 5
“However, existing human pose datasets focus on single-person actions [6, 25, 34, 44, 47, 54, 60, 67, 73], limiting the applicability to real-world scenarios.”
Yang, K., Tang, X., Wu, H., Qin, B., Liu, H., He, J., & Fan, Z. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755267 · Cited by 15
“For instance, Transformer-based models [11, 13, 33, 42] directly map musical features to pose parameters, offering strong temporal modeling capabilities but often producing motions that lack coherence and nat-uralness.”
InfluentialCross-modal deep generative models reveal the cortical representation of dancing
Takagi, Y., Shimizu, D., Wakabayashi, M., Ohata, R., & Imamizu, H. (2025) · Nature Communications · doi:10.1038/s41467-025-65039-w · Cited by 5
MotionScript: Natural Language Descriptions for Expressive 3D Human Motions
Yazdian, P. J., Lagasse, R., Mohammadi, H., Liu, E., Cheng, L., & Lim, A. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247385 · Cited by 5
“Furthermore, by training a text-to-motion synthesis model on MotionScript-augmented data, we demonstrate enhanced expressive power and improved generation of out-of-distribution motions compared to standard baselines.”
Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrations
Wang, Z., Zhuang, H., Li, L., Zhang, Y., Zhong, J., Chen, J., Yu, Y., Tang, B., & Wu, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i1.27783 · Cited by 6
HardMo: A Large-Scale Hardcase Dataset for Motion Capture
Liao, J., Luo, C., Du, Y., Wang, Y., Yin, X.-C., Zhang, M., Zhang, Z., & Peng, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00161 · Cited by 5
X's Day: Personality-Driven Virtual Human Behavior Generation
Haoyang, L., Wang, Z., Liang, W. G., & Wang, Y. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3549574 · Cited by 7
“Some researchers explore the generation of high-precision motions under various conditions, such as actions [36, 60], text [1, 15], audio [30, 34], and scene [39, 66].”
Stereo-Based 3D Human Pose Estimation for Underwater Robots Without 3D Supervision
Wu, Y.-K., & Sattar, J. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3557235 · Cited by 5
“In our implementation, we jointly train the model on the large-scale datasets AIST++ [30] and H3WB [31] to incorporate comprehensive 3D pose information.”
Cheng, Y., Liang, M., Huang, S., Han, G., Ning, J., & Liu, W. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00217 · Cited by 5
Perez-Martinez, R., Casas-Ortiz, A., & Santos, O. C. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3734180 · Cited by 6
“For evaluation of the generated movement, we will use several criteria [23]: objective metrics based on quality, diversity, and movement correlation or joint distance; supervised classification scores or time efficiency; ablation studies to justify the choice of key aspects of the model; or user…”
Liu, S., Zang, Z., Zhang, Z., Hu, S., & Xiao, B. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3610386 · Cited by 5
“These methods [35], [36], [37] mainly utilize the cross-attention mechanism in the Transformer to fuse di ff erent modalities in a pairwise fusion way.”
Music-to-Dance Generation with Optimal Transport
Wu, S., Lu, S., & Cheng, L. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/691 · Cited by 5
Enhancing Expressiveness in Dance Generation Via Integrating Frequency and Music Style Information
Huang, Q., He, X., Tang, B., Zhuang, H., Chen, L., Gao, S., Wu, Z., Huang, H., & Meng, H. M. L. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448469 · Cited by 5
“Recently, deep neural networks have emerged as a promising method for dance generation, including CNNs [9], RNNs [10, 11], GANs [12], Transformers [13, 14], and Diffusion models [15].”
BackgroundMethodEverything2Motion: Synchronizing Diverse Inputs via a Unified Framework for Human Motion Synthesis
Fan, Z., Ji, L.-B., Xu, P., Fan, S., & Chen, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27936 · Cited by 5
“Two tasks that have particularly benefited from these advancements are: text-to-motion (Guo et al. 2022a,b; Petro-vich, Black, and Varol 2022) and music-to-dance (Sun et al. 2020; Zhuang et al. 2022; Li et al. 2021) synthesis.”
BackgroundMethodInfluentialTexVocab: Texture Vocabulary-Conditioned Human Avatars
Liu, Y., Li, Z., Liu, Y., & Wang, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00169 · Cited by 7
“We split each sequence into 2 consecutive parts for training and testing, where the training part accounts for 40% ∼ 80% of the multi-view sequences, and the novel poses include the remaining part of the videos and other poses like poses from AIST++ dataset [23].”
MethodPose‐to‐Motion: Cross‐Domain Motion Retargeting with Pose Prior
Zhao, Q., Li, P., Wang, Y., Sorkine‐Hornung, O., & Wetzstein, G. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15170 · Cited by 8
DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis
Wang, Z., Li, J., Qin, X., Sun, S., Zhou, S., Jia, J., & Luo, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680980 · Cited by 5
“Initially, Crnkovic-Friis et al. [12], Li et al. [37], Tang et al. [56] tried to synthesize dance motions frame-by-frame with sequence-to-sequence models including RNN, LSTM, and Transformer.”
BackgroundMethodInfluentialFG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptions
Shi, X., Yao, W., Luo, C., Peng, J., Zhang, H., & Sun, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78104-9_30 · Cited by 18
TransSMPL: Efficient Human Pose Estimation with Pruned and Quantized Transformer Networks
Kim, Y., Yoo, H., Ryu, J.-H., Lee, S., Lee, J. H., & Kim, J. (2024) · Electronics · doi:10.3390/electronics13244980 · Cited by 5
fMRI2GES: Co-Speech Gesture Reconstruction From fMRI Signal With Dual Brain Decoding Alignment
Zhu, C., Shao, J., Lin, J., Wang, Y., Wang, J., Tang, J., & Li, K. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3558125 · Cited by 4
“Lastly, We utilize the Beat Consistency Score (BC) [81], [82] to quantify the synchronicity of rhythms and Diversity [83] to evaluate the range of distinctions in gestures generated from di ff erent inputs.”
How can we make robot dance expressive and responsive? A survey of methods and future directions
Wallace, B., Glette, K., & Szorkovszky, A. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1575667 · Cited by 6
Enhancing Dance-To-Music Generation via Negative Conditioning Latent Diffusion Model
Sun, C., Liu, G., Fleming, C., & Yan, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00779 · Cited by 6
You Think, You ACT: the New Task of Arbitrary Text to Motion Generation
Wang, R., Ma, C., Li, G., Xu, H., Li, Y., & Wang, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01117 · Cited by 4
“Human motion generation supports diverse multimodal inputs, including Action Texts [1, 11, 30, 38, 39, 47, 48, 50], action labels [4, 8, 25, 27, 29, 46],incomplete posture [6, 12, 39],music [2, 19, 20, 22, 36], images [34], and more [15, 23, 37, 42, 43].”
Computer Vision for Sports Analytics
Fujii, K. (2025) · SpringerBriefs in computer science · doi:10.1007/978-981-96-1445-5_2 · Cited by 13
A review of generative models for virtual human motion driving
Wang, P., Peng, H., Mao, S., Wang, S., & Tang, J. (2025) · Science China Information Sciences · doi:10.1007/s11432-023-4284-x · Cited by 4
Crnek, K., Močnik, G., & Rojc, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2531286 · Cited by 4
Gao, X., Yang, Y., Du, S., Qi, G.-J., & Han, J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3607810 · Cited by 4
“…can be grouped into different specific generative sub-tasks based on their different condition input types, such as start-end positions [19], [20], movement trajectories [38], [39], scene contexts [4], [18], textual descriptions [5], [40], background musics [41], [42], and speech audios [43], [44].”
DanceHAT: Generate Stable Dances for Humanoid Robots with Adversarial Training
Nie, B., & Gao, Y. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811649 · Cited by 4
Contrastive Self-Supervised Learning for Automated Multi-Modal Dance Performance Assessment
Zhong, Y., Zhang, F., & Demiris, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096824 · Cited by 4
HoloSinger: Semantics and Music Driven Motion Generation with Octahedral Holographic Projection
Jin, Z., Wang, Z., Wang, Q., Jia, J., Bai, Y., Zhao, Y. H., Li, H., & Wang, X. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612674 · Cited by 4
“Besides, our model has equivalent effects comparing with Bai-lando [4] (SOTA of Music2Motion) on objective evaluation indicators like FID ↓ ( 4.94 v.s. 5.21) and Beat Align Score [1] ↑ ( 0.2313 v.s. 0.1956), showing its competence for music driven choreography.”
InfluentialCross-Modal Quantization for Co-Speech Gesture Generation
Wang, Z. L., Zhang, W., Ye, L., Zeng, D., & Mei, T. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3405743 · Cited by 6
“Additionally, some works leverage music to generate dance [30], [31], [32] or violin performer movements [33], which exhibit strong rhythmicity and periodicity, making them well-inferred from music.”
BackgroundLi, P., Wang, Z., Liu, M., Liu, H., & Chen, C. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680597 · Cited by 4
Scalable Group Choreography via Variational Phase Manifold Learning
Le, N., Do, K., Bui, X., Do, T., Tjiputra, E., Tran, Q. D., & Nguyen, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_17 · Cited by 4
Dance2Music-Diffusion: leveraging latent diffusion models for music generation from dance videos
Zhang, C., & Yan, H. (2024) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-024-00370-6 · Cited by 4
MM-LDM: Multi-Modal Latent Diffusion Model for Sounding Video Generation
Sun, M., Wang, W., Qiao, Y., Sun, J., Qin, Z., Guo, L., Zhu, X., & Liu, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680889 · Cited by 10
“We conduct extensive experiments on the Landscape [14], AIST++ [15], and AudioSet [5] datasets, achieving new state-of-the-art generation performance with significant visual and auditory gains.”
InfluentialShape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions
Liao, T.-H., Zhou, Y., Shen, Y., Huang, C.-H. P., Mitra, S., Huang, J., & Bhattacharya, U. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00185 · Cited by 7
Music conditioned 2D hand gesture dance generation with HGS
Zhou, D., Liu, S., & Xu, Q. (2023) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2211 · Cited by 4
Multi-Modal Beat Alignment Transformers for Dance Quality Assessment Framework
Kim, T. (2024) · Journal of Multimedia Information System · doi:10.33851/jmis.2024.11.2.149 · Cited by 4
T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences
Lee, T., Baradel, F., Lucas, T., Lee, K. M., & Rogez, G. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00193 · Cited by 7
“Real-life human motion is continuous and can be viewed as a temporal composition of actions , with transition in be-tween.”
BackgroundRethinking Sampling for Music-Driven Long-Term Dance Generation
Truong-Thuy, T.-V., Bui-Le, G.-C., Nguyen, H.-D., & Le, T.-N. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0917-8_14 · Cited by 3
“In contrast, recent approaches treat dance generation as a motion synthesis problem, employing various network architectures, such as CNNs [11,19], RNNs [12,43], GANs [20,41], and Transform-ers [22–24].”
InfluentialControllable 3D Dance Generation Using Diffusion-Based Transformer U-Net
Guo, P., Hao, T., Fu, W. N., Gao, Y., & Li, Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i3.32339 · Cited by 4
“The researches on dance generation models develop along with the emergence of generative models, evolving from long short-term memory (LSTM) models (Tang, Jia, and Mao 2018; Tang, Mao, and Jia 2018; Ye et al. 2020; Chen et al. 2021; Zhuang et al. 2022) to Transformers (Li et al. 2020, 2021, 2022a).”
InfluentialMeloDance: Dance Generation Guided by Music Structure and Emotion
Li, Y., Jin, Q., Liu, H., Chen, J., Zhao, X., & Li, P. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733385 · Cited by 3
“Many rely on autoregressive models [18, 32], these models often accumulate prediction errors over time, hindering their ability to learn comprehensive choreographic patterns and resulting in poor movement consistency, especially in similar verses or choruses.”
FilmComposer: LLM-Driven Music Production for Silent Film Clips
Xie, Z., He, Q., Zhu, Y., He, Q., & Li, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01262 · Cited by 10
“Most video-to-music datasets only focus on generating music for human-centric videos—dance [23, 40, 44] or concert [26].”
DFM: Deep Fourier Mimic for Expressive Dance Motion Learning
Watanabe, R., Li, C., & Hutter, M. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128480 · Cited by 3
“While some previous studies have specifically focused on learning dancing motions [20], [21], the PAE [19] has proven effective in producing smoother and more stable movements for periodic motions, such as locomotion and dancing, with a special treatment of the spatial-temporal relationships…”
Suzuki, T., Tanaka, R., Yeung, C., & Fujii, K. (2025) · ACM International Workshop on Multimedia Content Analysis in Sports (MMSports) · doi:10.1145/3728423.3759402 · Cited by 3
“To address this gap, sports-specific datasets have emerged [10, 12, 14, 23, 32, 42].”
Artistic turing test: The challenge of differentiating human and AI-generated art
Cenerini, C., Keller, F., Pennazza, G., Santonico, M., & Vollero, L. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100209 · Cited by 5
Moon, J., Choi, Y., Hong, J.-H., & Kim, K.-J. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2596857 · Cited by 3
“The original model was trained based on the AIST þþ dataset (Li et al., 2021).”
Zhang, H., Li, Z., Qi, X., Li, M., Sun, M., Wang, S., Zhang, M., & Han, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01130 · Cited by 5
“DanceRemix Dataset: Since the existing dance generation datasets fail to provide accurate edit descriptions aligned with both dance motions and music, we contribute a new dataset named DanceRemix to evaluate our approach, upon prior work [17, 21, 25].”
MOVE: Motion-Guided Few-Shot Video Object Segmentation
Ying, K., Hu, H., & Ding, H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01082 · Cited by 5
“Videos in MOVE are sourced from two parts: i) public action recognition datasets [4, 9, 15, 23, 27, 29, 31, 32, 45, 70] and ii) internet videos under a Creative Commons License .”
Music-Aligned Holistic 3D Dance Generation via Hierarchical Motion Modeling
Li, X. F., Li, R., Fang, S., Xie, S., Guo, X., Zhou, J., Peng, J., & Wang, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01338 · Cited by 8
“Recent advancements in generative models, such as GAN-based and transformer-based frameworks, have improved the quality of generated dance [30, 31, 50, 60].”
InfluentialSelf-citationPark, S.-J., Choi, I., Soon, D., Jeon, Y., & Joo, K. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00177 · Cited by 3
“AIST++ consists of 1,408 high-quality short 3D dance sequences, performed by professional dancers across 10 genres.”
Influential
Metadata
- Authors
- Li, R., Yang, S., Ross, D. A., & Kanazawa, A.
- Affiliations
- University of Southern California (US)
- Google (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13381-13392
- OpenAlex
- W3204221554
- Cited by
- 553
- References
- 94
- Citations by year
- 2026: 92 · 2025: 146 · 2024: 154 · 2023: 113 · 2022: 46 · 2021: 2
- FWCI
- 117.49
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- cross-modal transformer, full attention, AIST, dance genres, choreography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
