The DOI matches its Crossref record; 90 of the 94 references it cites were found in a registry or an index.
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
Open accessAbstract
We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music. The proposed AIST++ dataset contains 5.2 hours of 3D dance motion in 1408 sequences, covering 10 dance genres with multi-view videos with known camera poses—the largest dataset of this kind to our knowledge. We show that naively applying sequence models such as transformers to this dataset for the task of music conditioned 3D motion generation does not produce satisfactory 3D motion that is well correlated with the input music. We overcome these shortcomings by introducing key changes in its architecture design and supervision: FACT model involves a deep cross-modal transformer block with full-attention that is trained to predict N future motions. We empirically show that these changes are key factors in generating long sequences of realistic dance motion that are well-attuned to the input music. We conduct extensive experiments on AIST++ with user studies, where our method outperforms recent state-of-the-art methods both qualitatively and quantitatively. The code and the dataset can be found at: https://google.github.io/aichoreographer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We note that the simple prediction-based late fusion [247], [248] is less adopted in MML Transformers.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [248]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 8×
“We propose Full Attention Cross-Modal Transformer (FACT) model for the task of 3D dance motion generation.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“All our experiments are trained with 16 batch size using Adam [45] optimizer.”
MethodJakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 1×
“Training data for 2D pose and audio is abundant thanks to the high reliability of 2D pose detectors [14].”
BackgroundDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 2×
“While there are many large scale 3D motion capture datasets [39, 59, 1, 37], mocap dataset of 3D dance is quite limited as it requires heavy instrumentation and expert dancers for capture.”
BackgroundMethod
Research works printed but not linked: 3
- lamothe, 2019, The dancing species: how moving together in time helps make us human, Aeon
- alemi, 2017, Groovenet: Real-time music-driven dance movement generation using artificial neural networks, Networks, 8, 26
- chen, 2021, Choreomaster: choreography-oriented music-driven dance synthesis, ACM Transactions on Graphics (TOG), 40, 1
Cited by
Sorted by citations
Items 401–500 of 553
Sun, Y., Luo, Y., & Chen, H. (2025) · IEEE Access · doi:10.1109/access.2025.3555297 · Cited by 1
Motion Generation Review: Exploring Deep Learning for Lifelike Animation with Manifold
Zhao, J., Weng, D., Du, Q., & Tian, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-3679-2_1 · Cited by 2
SyncViolinist: Music-Oriented Violin Motion Generation Based on Bowing and Fingering
Nishizawa, H., Tanaka, K., Hirata, A., Yamaguchi, S., Feng, Q., Hamanaka, M., & Morishima, S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00529 · Cited by 1
“Various meth-ods have been proposed, including the use of factored conditional restricted Boltzmann machines [2], transformers [20], and dilated convolution-based models [43].”
TalkinNeRF: Animatable Neural Fields for Full-Body Talking Humans
Chatziagapi, A., Chaudhuri, B., Kumar, A., Ranjan, R., Samaras, D., & Sarafianos, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91581-9_11 · Cited by 6
CDCGM: Composition-specified Dance Choreography Generation from Music
Ishii, R., Eitoku, S.-I., & Morency, L. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099188 · Cited by 1
“The content of our dataset is similar to that in AIST++ [14], a dance-motion dataset that is publicly available and was used in current previous research [1], [28], [24].”
InfluentialContextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation
Liu, P., Zhang, P., Kim, H., Garrido, P., Shapiro, A., & Olszewski, K. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755140 · Cited by 13
“…between real and generated gestures in feature space, Diversity (Div.) [25] to calculate the average feature distance between generated gestures, Beat Alignment Score (BAS) following [26], Percent of Correct Motion parameters (PCM) , difference of generation deviate from ground-truth following [6].”
PAMS-DP: Building a Unified Open PAMS Human Movement Data Platform
Tang, M., Huang, T., Luo, J., & Li, X. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9995069 · Cited by 1
“Google released the AIST++ dance movement dataset [16-17] in 2021, which contains 1408 3D human dance motion sequences and is the movement database with the most complete joint annotation.”
MethodNatural language control for 3D human motion synthesis
Petrovich, M. (2024) · theses.fr (ABES) · doi:10.70675/a4b835c3z7030z4699zbeb2zb5faccaee2de · Cited by 1
Next Level Choreography: Applying a Transformer Network to Generate Improvised Dance Motions
Asadi, Z., Moons, J., & Leijnen, S. (2023) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-28993-4_36
Brown, K., & Wei, D. (2023) · International Conference on Signal, Image Processing and Multimedia (SIPM) · doi:10.5121/csit.2023.130901
Video-Music Retrieval with Fine-Grained Cross-Modal Alignment
Era, Y., Togo, R., Maeda, K., Ogawa, T., & Haseyama, M. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222739
“25 , δ = 30 , and σ = 3 referring to the previous paper [13, 25].”
BackgroundMethodUniQNatyam: An Approach Towards Non-Repetitive Dance Generation
Murthy, V., Chandra, V., Hegde, V., Srishti, R., & Srinivas, K. (2023) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml59573.2023.10348205
“By integrating the Full Attention Cross-Modal Transformer (FACT) model [1] with seed motions and a reward model, we ensure unique and non-repetitive choreography.”
MethodTraDance: A dataset for assessing the performance of Greek folk dances recognition models
Balafoutis, T., Balaska, V. E., & Gasteratos, A. C. (2023) · IEEE International Conference on Imaging Systems and Techniques (IST) · doi:10.1109/ist59124.2023.10355720
“Initially, the work [7] presents a system that uses machine learning to generate 3D dance animations that are synchronized with music.”
BackgroundMethodA House Dance Design System Based on Constructive Choreographic Process
Oji, R., & Fujishiro, I. (2023) · The Journal of the Society for Art and Science · doi:10.3756/artsci.22.9_1
Research on Common Structure of Motion Data
Liu, J., Wang, H., Li, Y., & Li, P. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385523
“Google released the AIST++ dance movement dataset [14], [33] in 2021, which contains 1408 3D human dance motion sequences and is the movement database with the most complete joint annotation.”
MethodExploring Artificial Intelligence for Advancing Performance Processes and Events in Io3MT
Vieira, R., Muchaluat-Saade, D. C., & Cesar, P. S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53302-0_17
A Fast Sampling Method in Diffusion-Based Dance Generation Models
Guo, P., Han, Y., Gao, Y., & Li, Y. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-0601-3_10
Decomposed Human Motion Prior for Video Pose Estimation via Adversarial Training
Chen, W., Zhou, X., Yu, Z., Gu, W., & Zhang, K. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-53963-3_35
Let the Beat Follow You - Creating Interactive Drum Sounds From Body Rhythm
Liu, X., Su, K., & Shlizerman, E. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00702
“We also compute the Beat Alignment Score [37] which measures the correlation between motion and music.”
MethodPipa Performance Generation Based on Pre-trained Temporally Guided Network
Song, J., & Li, R. (2023) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3638884.3638978
“Several studies have attempted to generate body movements from audio and music signals, including generating the skeletons of pianists and violinists from audio [1, 2], gestures from conversational speech [3], movements from music conductors [4, 5]and dance movements from music [6–10].”
BackgroundMethodInfluentialSelf-citationden Hertog, J., & Saunders, R. (2024) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3658852.3659079
“Datasets with dance motions, AIST++ [4] and FineDance [5], contain high-quality recorded dance data paired with music and/or genre distinctions.”
BackgroundMethodAn Intelligent System for Chinese Dance Creation Using Generative Artificial Intelligence
Wang, L., Xu, S., & Lu, Y. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64315-6_43
Referring Expression Comprehension with Multi-Cross Decoder
Yi, Z., Feng, F. X., & Li, R. (2024) · International Conference on Computer and Automation Engineering · doi:10.1109/iccae59995.2024.10569910
“(3) Hierarchical Attention (multi-stream to one-stream) [31] can manage different modalities by first attending to elements within a modality (like objects in an image or words in a text) and then attending across modalities together to integrate the information.”
BackgroundSelf-citationABDGAN: Arbitrary Time Blur Decomposition Using Critic-Guided TripleGAN
Lee, T. B., & Heo, Y. S. (2024) · Sensors · doi:10.3390/s24154801
YNU-Dance: A Multimodal Ethnic Dance Action Dataset
Sun, S., Tang, Q., Liu, Y., Zhang, H., Song, Q., & Xu, D. (2024) · International Conference on Computing, Networks and Internet of Things (CNIOT) · doi:10.1145/3670105.3670151
“AIST[25]/AIST++[10] is currently one of the larger-scale dance datasets.”
Action Conditioned Attention Encoder-Decoder and Discriminator for Human Motion Generation
Bandi, C., & Thomas, U. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-66694-0_16
Zhang, L. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-2598
3D Human Animation Synthesis based on a Temporal Diffusion Generative Model
Cheng, B., Feng, W., Wu, Q., Chen, J., Cai, Z., Zhang, Y., Wang, S., & Che, B. (2024) · doi:10.1109/prmvia63497.2024.00028
Learning to Generate 3D-Aware Realistic Hand from 2D and 3D priors
Lü, K., Zhang, H., Zhu, D., & Yang, W. (2024) · Chinese Control Conference (CCC) · doi:10.23919/ccc63176.2024.10662843
“Variations of linear blend skinning techniques [5]have been applied to articulate radiance fields specifically tailored for single-scene scenarios [6], addressing the challenge of applying such deformation methods to generative models.”
BackgroundAudio2DiffuGesture: Generating a diverse co-speech gesture based on a diffusion model
Yao, H., Xu, Y., Weitao, W. U., He, H., Ren, W., & Cai, Z. (2024) · Electronic Research Archive · doi:10.3934/era.2024250
“To measure the correlation between synthetic gesture sequence and audio, Li et al. [44] proposed Beat Consistency Score (BC).”
BackgroundMethodDatasetNeRF: Efficient 3D-Aware Data Factory with Generative Radiance Fields
Chi, Y., Zhan, F., Wu, S., Theobalt, C., & Kortylewski, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73033-7_20
MMIDM: Generating 3D Gesture from Multimodal Inputs with Diffusion Models
Ye, J., Liu, C., Wan, H., Jiang, A., & Lei, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8508-7_22
Dance Generation From Music with Enhanced Beat
Wang, Y., & Li, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0119-6_15
AI-Based Music to Dance Synthesis and Rendering
Jin, B. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/2024.18318
CigTime: Corrective Instruction Generation Through Inverse Motion EditingPreprint
Fang, Q., Tang, C., Tekin, B., & Yang, Y. (2024) · Qeios · doi:10.32388/hiaxat
Multi-Head AdaIN Audio-to-Video Motion Style Transfer
Aldausari, N., Sowmya, A., & Mohammadi, G. (2024) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio64047.2024.10907605 · Cited by 1
“Without motion frames, models often use 2D or 3D keypoints to track motion [2], [1], [10], [11], [12], [13], [3], [8], [9], [15], [14], [16], [17], [20], [21], [22].”
Two-in-One: Unified Multi-Person Interactive Motion Generation by Latent Diffusion Transformer
Li, B., Wang, X., Song, R., & Huang, W. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887687 · Cited by 5
MotionComposer: Enhancing Rhythmic Music Generation with Adaptive Retrieval Reference
Wang, J., Liu, L., & Wang, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889094 · Cited by 1
“Following the previous work, we use the dance dataset AIST++ [32], the Figure skating dataset, and the floor exercise dataset collected in LORIS [5] to evaluate the effectiveness of our proposed method.”
Soft Computing Mobile App Build Dance Choreography Based on the Rhythm of Supplied Sound
Premkumar, A., Deepa, P., Kumar, Y. J. N., Almusawi, M. M., Venkataiah, V., & Ramya, M. (2025) · International Conference on Intelligent Control, Computing and Communications (IC3) · doi:10.1109/ic363308.2025.10956515
“The fact that dance is a form of art that can be utilised to communicate significant ideas, exchange cultures, assess society, and serve as a means of stress relief for dancers, choreographers, and spectators makes it relevant that dancers may encounter blocks in their creative process [6].”
Text-Guided Diverse Scene Interaction Synthesis by Disentangling Actions From Scenes
Teshima, H., Wake, N., Thomas, D., Nakashima, Y., Kawasaki, H., & Ikeuchi, K. (2025) · IEEE Access · doi:10.1109/access.2025.3562086
“…through diverse influences, ranging from historical motion trajectories [9], [10], [11], [12], [13], action categorizations [14], [15], [16], audio [17], [18], [19], [20], [21], [22], and textual descriptions [1], [8], [20], [21], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32].”
DanceDuo: Bridging Human Movement and AI Choreography
Bui-Le, G.-C., Truong-Thuy, T.-V., Nguyen, H.-D., & Le, T.-N. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4288-5_29 · Cited by 1
M2Beats 2.0: When motion meets beats in short-form videos twice
Zhang, Y., Lv, A., He, S., Zhang, H., & Ming, A. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111967
Umino, B., & Soga, A. (2025) · Nicograph International (NicoInt) · doi:10.1109/nicoint67466.2025.00022
“Research on automatic choreography generation using motion data has produced systems that generate movements based on music [8][9][10] or that add emotions to dance movements [11].”
Co-Speech Gesture Video Generation with Implicit Motion-Audio Entanglement
Li, X., Chen, Z., Yu, X., Chu, I.-H., Chang, P., & Xiao, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01063
Vividhead: Speech-Driven 3D Facial Animation with Head Pose Using Decoupled Speech Feature
Sima, L., Duan, Z., Feng, D., & Wang, Z. (2025) · IEEE International Workshop on Radio Frequency and Antenna Technologies (iWRF&AT) · doi:10.1109/iwrfat65352.2025.11103014
“FID Following Bailando [33], Ai choreographer [35], and Trimodal [36], we evaluate the quality of generated motions by calculating the distribution distance between the generated motions and ground truth (gt) motions using the Frechet Inception Distance (FID) on extracted motion features.”
Motion Style Transfer: Methods, Challenges, and Future Directions
Du, S., Cheng, B., Yi, W., Zhou, Z., & Yang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0100-7_4
Text-Driven 3D Human Motion Generation
Zhang, M., Cai, Z., Pan, L., Gu, C., Hong, F., Guo, X., Ren, J., Yang, L., & Liu, Z. (2025) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-031-94969-2_16
Music-Driven Legged Robots: Synchronized Walking to Rhythmic Beats
Hou, T., Zhang, Y., Wei, X., Dong, Z., Yi, J., Zhai, P., & Zhang, L. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127347
“Our music data comes from the AIST++ dataset[31].”
Synthesis of style-specific dance by learning body gestures from music
Islam, M. S., & Rahman, S. M. M. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-21123-3
“In general, transformer models are used to capture the long sequence 2D and 3D data through the self-attention mechanism [17].”
Lagrangian Motion Fields for Long-Term Motion Generation
Yang, Y., Huang, Z., Xu, C., & He, S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3612380
“While Lagrangian motion fields provide a structured theoretical framework for capturing motion dynamics in a discrete form, a practical mechanism is needed to obtain meaningful discretized motion segments.”
InfluentialMultimodal Quantitative Measures for Multiparty Behavior Evaluation
Shirekar, O., Pouw, W. T. J. L., Hao, C., Phadnis, V., Beeler, T., & Raman, C. (2025) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3716553.3750752
“To measure the consistency of timing between the different (EMD-derived) multimodal signals we utilize the beat consistency score as provided in [25], whereby the temporal alignment between signal onsets (i.e., beats) is calculated by a guassian-weighted proximity score (1 = perfect synchrony) that…”
Esaki, K., & Nagao, K. (2025) · International Conference Virtual and Augmented Reality Simulations · doi:10.1109/icvars66454.2025.11198714 · Cited by 1
“AIST++ [47] [48], a large dance movement dataset, was used for pre-training of each component of the proposed method because it contains various dance movements performed by multiple dancers and contains many dynamic human movement data as well as highly accurate pose annotations.”
Sanchez, J. I. G., & Costa, P. D. P. (2025) · GENEA · doi:10.1145/3746268.3759431
“4.1.3 Beat Alignment Score (BAS) [17].”
InfluentialRhythm Fusion: Synchronizing Audio and Motion Features for Music-Driven Dance Generation
Aldausari, N., Mohammadi, G., & Cooper, D. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-04339-9_7
Audio-Driven Gesture Generation via Deviation Feature in the Latent Space
Chen, J., Yang, H., Shi, R., Ding, C., Mo, X., Xiong, S., & He, Y. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209432
A Multi-Branch Network for Pose Trajectory Smoothing and Refinement
Chen, P., Jiang, Y., Hu, H., Wang, C., Li, H., & Gao, H. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11208983
“6M [35], AIST++ [36], and 3DPW [37] datasets to perform training and testing experiments.”
Style-Aware Music-to-Dance Generation via Multi-Stage Unit Decomposition and Recombination
Gao, Y., Wu, Q., He, K., & Zhou, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4384-7_7
Dynamic Masking Strategy: An Effective Approach to Enhancing Accurate Human Motion Generation
Liu, X., Zhang, Z., Qu, X., Yuan, S., Liu, Y., Chaomurilige, C., Liu, Z., & Jiang, S. L. (2025) · Journal of Artificial Intelligence Research and Innovation · doi:10.29328/journal.jairi.1001009
InterMamba: Efficient Human-Human Interaction Generation With Adaptive Spatio-Temporal Mamba
Wu, Z., Sun, Y., Chen, Y., Gu, X., Liu, R., & Chen, J. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3635116
“In recent years, with the rapid development of deep generative models[47],[50],[53],particularlydiffusionmodels[16],significant progresshasbeenachievedforsingle-personhumanmotiongen-erationintermsofcontrollability[20],[48],diversity[12],and quality[56].”
You Only Touch Once: One-Touch System for Personalized 3D Music Video Generation
Lee, K., Shin, Y., Huh, J., & Lee, S. (2025) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc65261.2025.11249334
Towards Accurate and Smooth Human Motion Reconstruction from Sparse Tracking with State Space Model
Dong, K., Xue, J., Niu, Z., Liu, Q., Mao, S., Lü, K., & Lan, X. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3778860
Dong, R., Ni, S., & Yang, X. (2025) · SIGGRAPH Asia Conference Papers · doi:10.1145/3757377.3764006
“4 (a), beat alignment score [Li et al. 2021], averages the closest temporal distances to assess the alignment between dance motions and musical beats.”
InfluentialEra, Y., Togo, R., Maeda, K., Ogawa, T., & Haseyama, M. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3645612
“Among them, multi-modal datasets have been notably proposed, such as motion-text datasets for the text-to-motion task [13], [18], [52], [53] and dance-music datasets for the dance generation task [50], [54], [55].”
InfluentialINTELLIGENT PERFORMANCE EVALUATION IN DANCE TRAINING
Goyal, E., Likhith, S. R., Kalele, G., Sharma, D. K., Behera, S. K., & Dingankar, S. (2025) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v6.i2s.2025.6752
Human Dance Generation Via Text-Music Integration
Zhang, Y. Y., Zhang, L., Hu, T., & Li, J. (2025) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm65671.2025.11293585
“Motion diversity is assessed by calculating the average characteristic distance of the generated movement to assess diversity [1].”
InfluentialMACHINE LEARNING FOR PREDICTING AUDIENCE PREFERENCES IN DANCE
Khan, F. A., Barisal, S. K., Thacker, C. B., Kaur, P., Devi, N., & Kulkarni, A. M. (2025) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v6.i5s.2025.6883
AI-Powered Music Generation from Sequential Motion Signals: A Study in LSTM-Based Modelling
Bukaita, W., Artiles, N., & Pathak, I. (2025) · International Journal of Intelligent Information Systems · doi:10.11648/j.ijiis.20251406.12
DreamDancer: Music-Driven Dance Video Intelligent Generation
Huang, D., Qian, J., Liu, Y., Tu, W., & Liu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5679-3_4
SDCGM: Dance Choreography Generation from Music with Dancer-specific Style
Ishii, R., Eitoku, S.-I., Fushio, K., Matsuo, S., Sato, Y., & Morency, L. (2026) · Journal of Information Processing · doi:10.2197/ipsjjip.34.2
Zhao, F. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1707539
МНОГОМОДАЛЬНАЯ ГЕНЕРАЦИЯ РЕЧИ, МИМИКИ И ЖЕСТОВ В ЦИФРОВЫХ АВАТАРАХ: СОВРЕМЕННЫЕ МЕТОДЫ И ПЕРСПЕКТИВЫ
Аксёнов, А. А. (2025) · Вестник ВГУ Серия Системный анализ и информационные технологии · doi:10.17308/sait/1995-5499/2025/4/155-182 · Cited by 1
DreamBarbie: Text to Barbie-Style 3D Avatars
Sun, X., Zhang, Z., Tai, Y., Tang, H., Yi, Z., & Yang, J. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3663253
LightingGen: A DMX Based Generation Method for Entertainment Stage Lighting
Wang, T., Jiang, Y., Jiang, W., Zhou, X., & Guan, X. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3664978
“A large-scale 3D dataset for music-driven dance, AIST++[40], based on the AIST dataset [41] has been later proposed, and they used a fully attention-based mechanism to generate 3D dance sequences from music.”
MambaGesture2: Co-Speech Gesture Generation via Hierarchical Fusion and Spatiotemporal Aggregation
Fu, C., Wang, Y., He, H., Wang, S., Wang, C., Tai, Y., Liu, Y., & Zhang, J. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668541
“Following the EMAGE framework, which introduced the BEAT2 dataset, we adopt FGD, L1Div, and BC [43] to evaluate gesture generation performance on both the BEAT2 and SHOW datasets.”
RIA-Net:Realistic Image Animation through Semantic-aware Feature Learning Network
Nega, A. T., Gedamu, K., Chen, A., & Chen, J. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00591-z
“Moreover, limited training data leads to poor zero-shot generalization to unseen scenarios, making it difficult to capture nuanced human attributes or complex poses (Li et al. 2021a; Liu et al. 2019).”
Wang, H., Li, J., Li, D., Zhang, K., & Sun, Z. (2025) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb65343.2025.11411217
“We adopt three objective metrics to evaluate the generated gestures from different perspectives: FGD [67] quantifies distributional similarity to real motions, BC [32] assesses rhythmic alignment with speech, and L1 Diversity [31] measures output variation across inputs.”
$\mathrm{W}^{2}$: Variable-Length Video-to-Music Generation
Gu, X., Jiang, W., Jiang, Y., Yan, M., & Su, Z. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3673504
“Data: Recently, video-to-music generation tasks have typically employed human-centric video music datasets, such as the AIST++ dance music dataset [44].”
GESTURE RECOGNITION FOR CLASSICAL DANCE FORMS USING COMPUTER VISION
Sen, P., Pathak, A., Gudadhe, M. B., Gudadhe, M. M., & Singh, V. (2026) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v7.i2s.2026.7221
Personalized Dance Synthesis Based on Physical and Cognitive Intensities
Tang, X., Yeo, E., Mao, R., Guo, X., & Alghofaili, R. (2026) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr67842.2026.00048 · Cited by 1
“This library was built by extracting and processing motion data from several publicly available datasets: AIST++ [30], AIOZ-GDANCE [29], FineDance [33].”
Harnessing the Power of Multimodal and Textual Data in Industry 4.0
Pellegrain, V. (2023) · theses.fr (ABES) · doi:10.70675/7522c20fza9b4z4c4az8d8dz290dd3669716 · Cited by 1
Multimodal deep learning for audiovisual production
Ouenniche, K. (2023) · theses.fr (ABES) · doi:10.70675/a01becf4z4ed9z4955z940cz49c821c1b8a8
Enhancing Human-Robot Interaction with Computer Vision
Du, Y. (2023) · theses.fr (ABES) · doi:10.70675/9c6ed094z8c91z435fz8111ze1ed33a7fe92
SynSP++: General Pose Sequences Refinement via Synergy of Smoothness and Precision
Jin, L., Wang, T., Xing, J., Zhao, J. C., & Li, X. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3682278
“AIST ++ dataset [50] is a new multi-modal dataset of rapid 3D dance motion and music, covering 10 dance genres with multi-view videos.”
Designing Movement Generation Models in Collaboration With Voguing And Dancehall Dancers
Chédin, L., Françoise, J., Caramiaux, B., & Alaoui, S. F. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791515
“Most studies introduce new models, which they typically evaluate based on a combination of quantitative metrics and crowd-sourced platforms such as Prolific, where participants are asked to select their preferred output [3, 22, 31, 48].”
InfluentialEMBODIED INTELLIGENCE AND ALGORITHMIC EXPRESSION: RETHINKING DANCE IN THE AGE OF AI
Zhang, L. (2026) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v7.i1.2026.7603
TEAMo: Trait and Emotion Aware Motion Generation in 3D Human
Tang, B., Zhu, D., Kuai, S., & Deng, C.-L. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464158
Visual Keys to Symphonies: Latent Diffusion for Multi-Scene Video-to-Music Generation
Ng, C. F., So, K., Yang, J., Ovalle, P., Lui, S., Fan, F., & Dong, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462058
Lu, Z., He, Z., Wang, Z., Li, Y., & Miao, Q. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462327
宣恩允, Li, Y., Li, Z., Yao, M., & Guo, R. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463428
Kobayashi, Tanoue, H., Suzuki, Y., Nishihara, H., Hori, T., Terada, T., & Tsuchida, S. (2026) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3802842.3802898
“We trained and evaluated our model on the AIST++ Dataset [12], a large-scale dance video dataset.”
Designing Generative AI for Real-Time Multi-User Interaction in Co-Creative Dance
Yang, M., Weber, R., & Lottridge, D. M. (2026) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3802842.3802865
“The original Bailando++ model [28] was trained solely on the AIST++ 3D dance motion dataset [13].”
EgoMusic-Driven Human Dance Motion Estimation with Skeleton Mamba
Nguyen, Q., Le, N., Huang, B., Vu, M. N., Tang, C., Nguyen, V., Le, N., Vo, T., & Nguyen, A. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01118 · Cited by 1
“With advancements in deep learning techniques and the availability of large-scale datasets, many networks have been introduced to generate higher-fidelity dance motions [2,15,27,37,40,43,67].”
InfluentialRapVerse: Coherent Vocals and Whole-Body Motion Generation from Text
Chen, J., Yan, X., Chen, Y., Cen, S., Wang, Z., Qx, M., Zhen, H., Qian, K., Lu, L., & Gan, C. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00941
“Finally, we adopt Beat Constancy (BC) [31] to measure the synchrony of generated motion and singing vocals.”
Freedance: Towards Harmonic Free-Number Group Dance Generation Via a Unified Framework
Zhao, Y., Wang, Y., Wen, L., Zhang, H., & Qi, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00983
“AIST++ [20] & AIOZ-GDANCE [17] FID With both CRAM and TIM, the model outperforms base-line in all metrics by a large margin.”
InfluentialMuseDance: A Diffusion-based Music-Driven Image Animation System
Dong, Z., Hao, W., Wang, J.-C., Zhang, P., & Polak, P. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00372
“The AIST++ [42] dataset, a subset of the AIST [64] dataset, includes both video and music modalities.”
A Computational Approach for Music-Driven Group Dance Formation Generation
Dissanayake, A., Samarakoon, K., Attanayake, J., Uthpalani, M., Krishara, J., & Wijendra, D. (2026) · International Research Conference on Smart Computing and Systems Engineering (SCSE) · doi:10.1109/scse70081.2026.11499816
“Each dancer acts as an independent agent that navigates from a current position to a target defined by subsequent formation [11].”
Su, M.-C., Huang, W.-L., & Pan, T. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02311-z
“For the music-to-dance generation task (Section 4.2), we compare our DanceLDM with previously discussed dance generation methods [ 2 , 3, 14 , 15 , 32].”
InfluentialRPGD: RANSAC-P3P Gradient Descent for Extrinsic Calibration in 3D Human Pose Estimation
Tuo, Z. (2026) · International Conference on Advances in Artificial Intelligence and Machine Learning (AAIML) · doi:10.1109/aaiml67890.2026.11498194
“6M [4], AIST++ [5], and an in-the-wild capture setup, demonstrating that RPGD consistently recovers extrinsic parameters with accuracy comparable to the original ground truth.”
Metadata
- Authors
- Li, R., Yang, S., Ross, D. A., & Kanazawa, A.
- Affiliations
- University of Southern California (US)
- Google (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13381-13392
- OpenAlex
- W3204221554
- Cited by
- 553
- References
- 94
- Citations by year
- 2026: 92 · 2025: 146 · 2024: 154 · 2023: 113 · 2022: 46 · 2021: 2
- FWCI
- 117.49
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- cross-modal transformer, full attention, AIST, dance genres, choreography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
