The DOI matches its Crossref record; 90 of the 94 references it cites were found in a registry or an index.
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
Open accessAbstract
We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music. The proposed AIST++ dataset contains 5.2 hours of 3D dance motion in 1408 sequences, covering 10 dance genres with multi-view videos with known camera poses—the largest dataset of this kind to our knowledge. We show that naively applying sequence models such as transformers to this dataset for the task of music conditioned 3D motion generation does not produce satisfactory 3D motion that is well correlated with the input music. We overcome these shortcomings by introducing key changes in its architecture design and supervision: FACT model involves a deep cross-modal transformer block with full-attention that is trained to predict N future motions. We empirically show that these changes are key factors in generating long sequences of realistic dance motion that are well-attuned to the input music. We conduct extensive experiments on AIST++ with user studies, where our method outperforms recent state-of-the-art methods both qualitatively and quantitatively. The code and the dataset can be found at: https://google.github.io/aichoreographer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We note that the simple prediction-based late fusion [247], [248] is less adopted in MML Transformers.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [248]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 8×
“We propose Full Attention Cross-Modal Transformer (FACT) model for the task of 3D dance motion generation.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“All our experiments are trained with 16 batch size using Adam [45] optimizer.”
MethodJakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 1×
“Training data for 2D pose and audio is abundant thanks to the high reliability of 2D pose detectors [14].”
BackgroundDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 2×
“While there are many large scale 3D motion capture datasets [39, 59, 1, 37], mocap dataset of 3D dance is quite limited as it requires heavy instrumentation and expert dancers for capture.”
BackgroundMethod
Research works printed but not linked: 3
- lamothe, 2019, The dancing species: how moving together in time helps make us human, Aeon
- alemi, 2017, Groovenet: Real-time music-driven dance movement generation using artificial neural networks, Networks, 8, 26
- chen, 2021, Choreomaster: choreography-oriented music-driven dance synthesis, ACM Transactions on Graphics (TOG), 40, 1
Cited by
Sorted by citations
Items 301–400 of 553
MACE-Dance: Motion-Appearance Cascaded Experts for Music-Driven Dance Video Generation
Yang, K., Zhu, J., Tang, X., Peng, Z., Zhang, X., Wang, P., Wu, J., Chu, X., Liu, H., & He, J. (2026) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3799902.3811202 · Cited by 23
GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstruction
Ma, S., Cao, Q., Yi, H., Zhang, J., & Tao, D. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612276 · Cited by 3
“capability to accurately predict 3D human motion with physical plausibility from RGB videos, which encompasses realistic interactions with the ground, is crucial for numerous applications [46], such as scene understanding [8, 19, 44], 3D dance motion reconstruction and generation [5, 20, 21, 39], and augmented and vir…”
BackgroundLearning Pose Controllable Human Reconstruction With Dynamic Implicit Fields From a Single Image
Li, J., Liu, X., & Lu, G. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3363493 · Cited by 3
Audio2Moves: Two-Level Hierarchical Framework for Audio-Driven Human Motion Synthesis
Cheng, Y., Elmasry, N., & Wang, Y. (2025) · SN Computer Science · doi:10.1007/s42979-025-03910-9 · Cited by 3
“We adopt the Beat Alignment Score used in previous work [42, 49, 52, 67, 107], as a metric to assess the synchrony between gesture and speech.”
Fan, W., & An, X. (2025) · Scientific Reports · doi:10.1038/s41598-025-21266-1 · Cited by 3
MoCo‐Flow: Neural Motion Consensus Flow for Dynamic Humans in Stationary Monocular Cameras
Chen, X., Li, W., Cohen‐Or, D., Mitra, N. J., & Chen, B. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14465 · Cited by 3
HyperCUT: Video Sequence from a Single Blurry Image using Unsupervised Ordering
Pham, B.-D., Tran, P., Tran, A. N., Pham, C. H., Nguyen, R. M. H., & Hoai, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00949 · Cited by 3
Maiorca, A., Bohy, H., Yoon, Y., & Dutoit, T. (2023) · ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) · doi:10.1145/3623264.3624443 · Cited by 5
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models
Yin, W., Tu, R., Yin, H., Kragić, D., Kjellström, H., & Björkman, M. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309317 · Cited by 4
“Inspired by Li et al. [25] and Guillermo et al. [1], our model outputs the next T p poses given the past T h poses to improve model performance.”
MethodDing, S., Hou, X., Liu, Y., Zhu, W., Fang, D., Fan, Y., Li, K., Huang, L., & Zhou, F. (2023) · Electronics · doi:10.3390/electronics12224696 · Cited by 3
Denoising Diffusion Probabilistic Models for Action-Conditioned 3D Motion Generation
Zhao, M., Liu, M., Ren, B., Dai, S., & Sebe, N. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446185 · Cited by 4
“…methods can be roughly divided into two categories: 1) unconditioned generation [5] and 2) conditioned synthesis with the condition coming from various modalities i.e. , music [6], emotion [7], pre-defined action categories [8, 9], text prompts [10, 2], past and/or future pose [11, 12, 13].”
BackgroundLee, S., Kang, W., Hong, J.-H., & Kong, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2360773 · Cited by 3
“We will then consider the additional adaptation of a number of useful evaluation metrics used in many recent dance-focused studies (Li et al., 2021, 2023; Zhou et al., 2023).”
Zhuang, Y., Cai, Y., Zhang, W., & Jin, C. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681528 · Cited by 3
“Human motion prediction is a crucial task, which predicts future motion trends based on the previous observations, and has wide applications in the fields of 3D character animation [5, 7, 25, 44, 46], surveillance systems [6, 22, 52], and autonomous driving [8, 24, 50].”
CEDT2M: text-driven human motion generation via cross-modal mixture of encoder-decoder
Wang, X., Li, P., & Wang, R. (2025) · Neural computing & applications (Print) · doi:10.1007/s00521-025-11112-w · Cited by 2
“On the other hand, conditioned synthesis focuses on achieving controllability by leveraging various conditioning factors, including music [28–30], speech [31, 32], action [33, 34], and text [35–40].”
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
Bai, Z., Bagautdinov, T., Romero, J., Zollhöfer, M., Tan, P., & Saito, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20068-7_13 · Cited by 3
Flow2Flow: Audio-visual cross-modality generation for talking face videos with rhythmic head
Wang, Z., He, W., Wei, Y., & Luo, Y. (2023) · Displays · doi:10.1016/j.displa.2023.102552 · Cited by 5
MIDGET: Music Conditioned 3D Dance Generation
Wang, J., Mao, W., & Liu, M. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8388-9_23 · Cited by 7
“Evaluation Metrics and Baslines We adopt the Fr ´ echet Inception Distances (FID) [5, 17, 24] to evaluate our method by following existing works.”
BackgroundMethodResultInfluentialReChoreoNet: Repertoire-based Dance Re-choreography with Music-conditioned Temporal and Style Clues
Au, H. Y., Chen, J., Jiang, J., & Guo, Y. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1478-9 · Cited by 2
Birds of a Feather: Learning to Retrieve Dance Poses From Music Via Ground-Truth Annotation Lifting
Tseng, B.-W., Wei, W.-L., & Lin, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890717 · Cited by 2
“…and integrated it with adapter-based fine-tuning techniques [12] to advance music-to-dance retrieval, we found that the benchmark dataset AIST++ [9] lacks discriminative power in distinguishing between different dance genres within the 3D pose and shape annotations, which in turn limits the…”
Controlling Human Shape and Pose in Text-to-Image Diffusion Models via Domain Adaptation
Buchheim, B., Reimann, M., & Döllner, J. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00363 · Cited by 3
“We assess visual fidelity by using poses and prompts from an evaluation set of annotated images to generate similar images, against which we benchmark fidelity metrics.”
InfluentialZheng, J., Cao, M., & Zhang, C. (2025) · IEEE Access · doi:10.1109/access.2025.3572954 · Cited by 3
“Though used for music-dance learning, the AIST++ [38] and TikTok [39] datasets have limited music content and provide only audio formats.”
Inter - Diffusion Generation Model of Speakers and Listeners for Effective Communication
Huang, J., Cheng, Y., Yu, M., Han, G., Li, J., Zhang, J., Wang, S., & Gu, X. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733366 · Cited by 2
“Beat Alignment (BA) [13] evaluates the synchronization between generated gestures and audio beats.”
Zhou, Z., Mei, K., Lu, Y. F., Wang, T., & Rao, F. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00300 · Cited by 3
“Some datasets [4, 26, 35–39] provide annotations for video-music rhythm matching, such as BGM909 [4] providing short music descriptions, music chords, and beats, but they lack analysis of emotional alignment and semantic transitions.”
M2PE-Diff: Music-to-Pose Encoder for Dance Video Generation Leveraging Latent Diffusion Framework
Park, N. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754808 · Cited by 2
“SSIM evaluates structural consistency between generated and ground-truth frames, PSNR To assess the synchronization between music and dance motion, we adopt the 2D-BeatAlign score, inspired by the BeatAlign metric defined in [ 16].”
MotionFlow: Efficient Motion Generation With Latent Flow Matching
Dong, K., Xue, J., Lan, X., Liu, Q., & Lü, K. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3651041 · Cited by 2
“To ease the creative process, recent research [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11] has focused on generating human motions in a natural interactive manner.”
Maiorca, A., Yoon, Y., & Dutoit, T. (2023) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3573381.3596460 · Cited by 2
“More recently, in music-driven motion generation model, virtual characters learn to perform a dance that matches with the input music and user studies have been performed to measure the motion samples rhythmic and aesthetic consistency [1, 15, 21, 23, 48].”
BackgroundHuMoMM: A Multi-Modal Dataset and Benchmark for Human Motion Analysis
Zhang, X., Wang, M., Zeng, M., Kang, W., & Deng, F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46305-1_17 · Cited by 2
Virtual Reality Dance Tracks from Skeletal Animations
Mastra, L., Silva, L. J. S., Raposo, A. B., & da Silva, V. (2023) · Symposium on Virtual and Augmented Reality (SVR) · doi:10.1145/3625008.3625035 · Cited by 2
“[20] is a music-conditioned 3D dance generation neural network, based on the also proposed dataset AIST++. Choreomaster [6] generates a dance motion sequence to accompany input music in terms of style, rhythm and structure.”
BackgroundMusic Conditioned Generation for Human-Centric Video
Zhao, Z., Zuo, B., & Wang, Y. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3358978 · Cited by 3
“Datasets: AIST++ [30] is used to train our learning modules for 150 epochs.”
MethodCo-speech Gesture Generation with Variational Auto Encoder
Ka, S., & Shinoda, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53311-2_12 · Cited by 2
Gesture Generation Via Diffusion Model with Attention Mechanism
Li, L., Weicong, L., Ding, Q., Tang, C., & Wang, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448118 · Cited by 2
“We employ three metrics, i.e., Fr ´ echet Gesture Distance (FGD) [6], Semantics-Relevant Gesture Recall (SRGR) [9], and BeatAlign [18], to quantitatively measure the model performance.”
MethodMusic-Driven Choreography Based on Music Feature Clusters and Dynamic Programming
Lin, S., Moshe Zukerman, D., & Yan, H. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3390232 · Cited by 2
“[9].”
BackgroundMethodInfluentialMusic-driven Character Dance Video Generation based on Pre-trained Diffusion Model
Xu, J., Liu, C., Cai, J., Ye, J., Lei, Z., & Jiang, A. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650092 · Cited by 2
“In music-driven dance generation tasks, most works focus on generating the motion joint point sequences [4]–[10], [10]– [13].”
BackgroundMethodA Transfer Learning Approach for Music-driven 3D Conducting Motion Generation with Limited Data
Oh, J. S., Jeong, J., & Chai, Y. H. (2024) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3641825.3689531 · Cited by 2
“Beat Alignment Score, an evaluation metric proposed by [2], measures the music-motion correlation, with higher values indicating a stronger correlation.”
L la ni ma tion: L lama Driven Gesture Animation
Windle, J., Matthews, I., & Taylor, S. L. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15167 · Cited by 3
Generalizing Human Motion Style Transfer Method Based on Metadata-independent Learning
Era, Y., Togo, R., Maeda, K., Ogawa, T., & Haseyama, M. (2024) · SIGGRAPH Asia Posters · doi:10.1145/3681756.3697890 · Cited by 2
DiffListener: Discrete Diffusion Model for Listener Generation
Jung, S., & Kim, T. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890674 · Cited by 5
FastTalker: An unified framework for generating speech and conversational gestures from text
Zhang, J., Guo, Z., He, M., & Yoshie, O. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130074 · Cited by 3
Liu, Z., Lou, S., Feng, Y., Huang, W., Hu, B., Lu, C., & Tan, J. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103392 · Cited by 3
Semantics-aware human motion generation from audio instructions
Wang, Z.-A., Zou, S., Yu, S., Zhang, M., & Dong, C. (2025) · Graphical Models · doi:10.1016/j.gmod.2025.101268 · Cited by 2
“This underscores the unique advantage of audio in improving both the ease and depth of human-computer interaction.”
FastTalker: Jointly Generating Speech and Conversational Gestures from Text
Guo, Z., & Zhang, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93806-1_14 · Cited by 5
When Less is More: A Sparse Facial Motion Structure for Listening Motion Learning
Nguyen, T., Dam, Q. T., Tran, D. T., & Lee, J. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3569889 · Cited by 2
“…facial motion sequences are continuous by nature, recent advancements in speech, natural language processing, and human motion synthesis [6], [7], [8] have led to the adoption of discrete motion tokens [9], [10], [11], rather than continuous representations [12], [13], for capturing human…”
Dual Reciprocal Learning of Language-based Human Motion Understanding and Generation
Liang, C., Shi, Z., Wang, W., & Yang, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00590 · Cited by 2
Forecasting Bimanual Object Manipulation Sequences from Unimanual Observations
Razali, H., & Demiris, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28266 · Cited by 2
MotionDreamer: Exploring Semantic Video Diffusion Features for Zero-Shot 3D Mesh Animation
Uzolas, L., Eisemann, E., & Kellnhofer, P. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00087 · Cited by 15
“The pose fitting performance of WHAM [75] and variants of our method for re-rendered AIST++ human body sequences [37].”
Speech-driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature InferencePreprint
Zhang, F., Wang, Z., Lyu, X., Zhao, S., Li, M., Geng, W., Ji, N., Du, H., Gao, F., Wu, H., et al. (2023) · doi:10.36227/techrxiv.24268702 · Cited by 2
DAMUS: A Collaborative System for Choreography and Music Composition
Zhou, T., Yu, B., Min, J., & Wang, Z. (2022) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew56448.2022.9859441 · Cited by 1
“State-of-the-art datasets like AIST++ and deep learning architecture like transformers also pushed the boundaries of dance synthesis, producing realistic dance motions matching the input music [9, 10].”
BackgroundGraph Convolutional GRU for Music-Oriented Dance Choreography Generation
Hou, Y., Yao, H., & Li, H. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00361 · Cited by 1
“There are also some works [18]–[21] devoted to the application of music-conditioned 3D dance generation.”
BackgroundDiT-Gesture: A Speech-Only Approach to Stylized Gesture Generation
Zhang, F., Wang, Z., Lyu, X., Ji, N., Zhao, S., & Gao, F. (2024) · Electronics · doi:10.3390/electronics13091702 · Cited by 1
Cospeech body motion generation using a transformer
Lu, Z., He, Z., Hong, J., & Ping, G. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05769-4 · Cited by 1
“As defined in [9], all three transformer encoders use a full-attention mask.”
Target Pose Guided Whole-body Grasping Motion Generation for Digital Humans
Shao, Q., & Fang, Y. (2024) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm62033.2024.10715927 · Cited by 1
“In this work, we propose a grasping motion generation framework for digital human which is an anthropomorphic intelligent agent with high degrees of freedom in virtual world.”
MethodToward Physically Stable Motion Generation: A New Paradigm of Human Pose Representation
Cui, Q., Lou, Z., Song, Z., & Shu, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3518054 · Cited by 1
“2) Baselines: We compare four benchmark methods from recent literature: FACT [58], DanceNet [59], Bailando [60], and EDGE [61].”
InfluentialMDD: Masked Deconstructed Diffusion for 3D Human Motion Generation from Text
Chen, J., Liu, F., & Wang, Y. (2025) · IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) · doi:10.1109/aixvr63409.2025.00017 · Cited by 1
“Similarly, music-driven dance choreography research explored CNN [36], [75], [84], VAE [55], RNN [2], [65], LSTM [7], [15], [32], AdaIN [7] and transformers [13], [14], [39], [40], [43], [63] to synthesize dance performances given music inputs.”
Accurate baseball player pose refinement using motion prior guidance
Oh, S., & Kim, H. (2025) · ICT Express · doi:10.1016/j.icte.2025.03.008 · Cited by 1
Discrete diffusion model with contrastive learning for music to natural and long dance generation
Wang, H., Jiang, Y., Zhou, X., & Jiang, W. (2025) · npj Heritage Science · doi:10.1038/s40494-025-01668-0 · Cited by 2
Beat-aligned motor synergies and kinematic beat detection in street dance movements
Shen, K., & Hirayama, J.-I. (2025) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-025-01626-8 · Cited by 1
“In a previous study [21], the alignment between music beats and kinematic beats was evaluated with the beat-alignment (BA) score, given by (5) Here, we present a novel approach to detecting kinematic beats based on synergy activations.”
InfluentialSan, L. W., Ajibade, S.-S. M., Jasser, M. B., Adediran, A. O., & Ayodele, A. (2025) · IEEE International Conference on Robotics and Technologies for Industrial Automation (ROBOTHIA) · doi:10.1109/robothia63806.2025.10986570 · Cited by 1
Keystate-Driven Long-Term Generation of Bimanual Object Manipulation Sequences
Razali, H., & Demiris, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3573081 · Cited by 1
“It can be conditioned on variables such as scene [29], audio [30], text [31], semantic action labels [11], [12], and objects [6], [32].”
MTR-MSE: Motion-Text Retrieval Method Based on Motion Semantics Expansion
Xu, D., Zheng, T., Zhang, Y., Yang, X., & Fu, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130632 · Cited by 1
BVI-CR: A Multi-View Human Dataset for Volumetric Video Compression
Gao, G., Azzarelli, A., Kwan, H. M., Anantrasirichai, N., Zhang, F., Andrew, W., Moolan-Feroze, O., & Bull, D. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11044241 · Cited by 1
MOST: Motion Diffusion Model for Rare Text via Temporal Clip Banzhaf Interaction
Wang, Y., Li, M., Leng, Z., Li, F. W. B., & Liang, X. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3588509 · Cited by 1
“The multimodal input data include music [7], [8], [9], [10], [11], [12], [13], motion categories [14], [15], [16], [17], motion trajectories [18], [19], [20], and text [21], [22], [23], [24], [25], [26].”
DiffCap: Diffusion-Based Real-Time Human Motion Capture Using Sparse IMUs and a Monocular Camera
Pan, S., Yi, X., Zhou, Y. J., Jian, W., Zhang, Y., Wan, P., & Xu, F. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3596403 · Cited by 1
Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic
Tang, J., Yang, H., Chen, T., & Hu, J.-F. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00182 · Cited by 1
Jiang, J., Xiao, W., Lin, Z., Zhang, H., Ren, T., Gao, Y., Lin, Z., Cai, Z., Yang, L., & Liu, Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02504 · Cited by 14
“Following Duolando [65], we obtain FID and diversity using motion features from AIST++ [43].”
Yi, L., Chen, C., & Yi, L. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00136 · Cited by 1
“There are some works[26, 28, 30] that generate human motion conditioned on music and speech.”
FORCE: Physics-Aware Human-Object Interaction
Zhang, X., Bhatnagar, B. L., Starke, S., Petrov, I. A., Guzov, V., Dhamo, H., Pérez-Pellitero, E., & Pons‐Moll, G. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00139 · Cited by 7
“However, earlier approaches either disregard the surrounding environment [60, 69, 70, 80] or focus solely on static objects [30, 74, 97, 111, 131, 132].”
Text-Conditioned Zero-Shot 3D Avatar Creation and Animation
Hong, F., Zhang, M., Pan, L., Cai, Z., Yang, L., & Liu, Z. (2025) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-031-94969-2_15 · Cited by 1
Goal-Driven Human Motion Synthesis in Diverse Tasks
Hwang, I., Bae, J., Lim, D., & Kim, Y. M. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00275 · Cited by 1
“For example, several works allow user to define the input conditions for motion synthesis, such as text [3, 15, 16, 37, 44, 45, 59, 69, 71, 73], music [35, 47, 51, 60] or paired object trajectories [5, 13, 33, 34, 54, 67].”
Li, X., Lin, J., Zhang, B., Qi, Y., Wang, C., & He, G. (2025) · Computer Graphics Forum · doi:10.1111/cgf.70261 · Cited by 2
Permission to Dance: An End-to-End Dance Enhancement System from Dance Capture to Analysis
Kim, J., Huh, J., Park, Y., Kim, S., Choi, J., & Lee, S. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754480 · Cited by 1
“We leverage EDGE[13] as a pretrained diffusion model, trained on a professional dance dataset AIST++[7].”
Learning Human Motion with Temporally Conditional Mamba
Nguyen, V.-Q., Le, Q.-T., Huang, B., Vu, M. N., Le, N., Vo, T. N., & Nguyen, A. (2025) · SIGGRAPH Asia Conference Papers · doi:10.1145/3757377.3763948 · Cited by 1
“Since our task leverages both egocentric video and music, we ensure fair comparison by equipping egocentric baselines with the Jukebox [Dhariwal et al. 2020] encoder for audio processing.”
Dance3DRelight: Single-Image 3D Reconstruction and Physically-Based Relighting of Dance Performances
Jamil, S., Muselet, D., Trémeau, A., & Colantoni, P. (2025) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta68720.2025.11302460 · Cited by 1
AI-Assisted Character Motion Synthesis using Deep Generative Models
Thapliyal, M., Jahan, A., & Sharma, R. (2025) · International Conference on IoT Based Control Networks and Intelligent Systems (ICICNIS) · doi:10.1109/icicnis66685.2025.11315783 · Cited by 1
“As for dance and music-conditioned generation, the progress was fast, thanks to the AIST++ / AI-Choreographer release, followed by DanceFormer, EDGE, TM2D, MIDGET, and others have introduced new datasets, architectures, and specialized metrics for beat-alignment and stylistic control [13][14].”
CrowdMoGen: Event-Driven Collective Human Motion Generation
Cao, Y., Guo, X., Zhang, M., Xie, H., Gu, C., & Liu, Z. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02677-z · Cited by 1
“…2022; Yan et al., 2023; Wang et al., 2023; Mao et al., 2020; Barsoum et al., 2018; Chen et al., 2023a; Jiang et al., 2023a; Sun and Chowdhary, 2023), for example, generate complete sequences from incomplete historical data or partial movements, ensuring the generated motions are smooth and…”
An explainable real time sensor graph transformer for dance recognition
Han, J., Wang, S., & Gao, J. (2026) · Scientific Reports · doi:10.1038/s41598-025-34691-z · Cited by 1
The “Digital Dance Innovation Project” Case
Liu, W., Shao, H., Yang, J., Li, X., & Li, G. (2026) · Journal of Cases on Information Technology · doi:10.4018/jcit.402406 · Cited by 1
DNF-Avatar: Distilling Neural Fields for Real-time Animatable Avatar Relighting
Jiang, Z., Wang, S., & Tang, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00668 · Cited by 1
Dance to Music Generation Based on Residual Vector Quantization
Lin, S., Moshe Zukerman, D., & Yan, H. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402456 · Cited by 1
“We train our model using the AIST++ dataset [7] for dance.”
InfluentialExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis
Zhou, X., Li, F., Chen, M., Zhou, Y., Wan, P., Jin, Y., Zhang, H., Liu, H., Fan, Z., He, J. H., et al. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3679469 · Cited by 2
“For generation quality, we adopt the following evaluation metrics: Fréchet Gesture Distance (FGD) [31], Beat Consistency (BC) [29], and Diversity (Div) [26].”
Multi-Genre Sports Dance Action Recognition and Skill Assessment Using CNN-BiLSTM-Attention Model
Zou, S. (2025) · International Conference on Computer Science and Management Technology (ICCSMT) · doi:10.1145/3795154.3795215 · Cited by 1
“Dance, as a sophisticated form of athletic expression, combines complex motor patterns with artistic interpretation, making it an ideal domain for studying human movement quality and expertise development [2].”
FODGE : High-Fidelity Dance Generation via Full-Body Optimization
Huang, X., Zhang, S., Zhang, Q. J., Guo, X., & Ye, L. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463867 · Cited by 1
PS-Mamba: Spatial-Temporal Graph Mamba for Pose Sequence Refinement
Dong, H., & Lee, G. H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00802 · Cited by 1
“6M [ 23] and AIST++ [32, 47] datasets.”
InfluentialMDD: A Dataset for Text-and-Music Conditioned Duet Dance Generation
Gupta, P., Fotso-Puepi, J. A., Li, Z., Mehta, J., & Bera, A. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01293 · Cited by 1
“AIST++ [14] remains a widely used benchmark with five hours of 3D motion across ten genres.”
Macaluso, G., Mandelli, L., Bicchierai, M., Berretti, S., & Bagdanov, A. D. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00101 · Cited by 1
“Recent breakthroughs in generative modeling, particularly denoising diffusion probabilistic models [3, 22, 42, 44, 50], have elevated the quality and fidelity of synthesized human motion.”
Recent advances in music generation: methods, evaluation, and challenges
段玮, Yu, Y., & Briot, J.-P. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11582-x · Cited by 2
Chen, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30058-4_1 · Cited by 1
Content-Rhythm Awareness Contrastive Learning for Music-Driven 3D Dance Generation
Guo, X., Zhao, Y., & Li, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02961-6 · Cited by 1
“…2019; Sun et al., 2020), GCNs (Ferreira et al., 2021; Ren et al., 2020), VQ-VAE (Gong et al., 2023; Li et al., 2022b; Zhang et al., 2023c), Transformers (Kim et al., 2022; Li et al., 2020, 2021b, 2022) and Stable-Diffusion (Dabral et al., 2023; Li et al., 2023; Qi et al., 2023; Tseng et al., 2023).”
InfluentialArtGDance: music-driven 3D group dance generation framework
Zhang, Y., Duan, R., Yu, H., & Yang, K. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21851-0 · Cited by 1
“Existing methods based on Generative Adversarial Networks (GANs) [11, 12, 26], Variational Auto-encoders (VAEs) [13 ], Diffusion Models [ 14–17, 27– 29], and Transformer-or Mamba-based Mamba-based architectures [ 18 , Despite these advances, most solo music-to-dance methods primarily focus on the…”
InfluentialVersatileMotion: A Unified Framework for Motion Synthesis and Comprehension
Ling, Z., Han, B., Li, S., Cheng, J., Shen, H. F., & Zou, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37335-9_19 · Cited by 1
TG-Dance: TransGAN-Based Intelligent Dance Generation with Music
Huang, D., Zhang, Y., Li, Z., & Liu, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-27077-2_19 · Cited by 1
DAncing body, Speaking Hands (DASH): Sign Dance Generation System with Deep Learning
Kim, E., Park, T., Moon, J., You, W., Ha, T.-G., & Kim, K.-J. (2023) · ACM SIGGRAPH Posters · doi:10.1145/3588028.3603687 · Cited by 1
“2022] with AIST++ [Li et al. 2021] 3D dance dataset.”
MethodDanceU: motion-and-music-based automatic effect generation for dance videos
Pan, Y., Du, Y., Wang, S., Ye, Y., Jiang, Y., Zhou, Z., Xu, L., Lu, M., Lin, Y., & Lu, J. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00358 · Cited by 1
Path guided motion synthesis for Drosophila larvae
Chen, J., Wang, Y., Sun, Y., Yu, Y., Liu, Z., Gong, Z., & Zheng, N. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200529 · Cited by 1
“The mean per-joint position error (MPJPE) is a commonly used metric to evaluate the accuracy of estimated poses (Ionescu et al., 2014; Li RL et al., 2021).”
BackgroundMethodInfluentialMusic-to-Dance Poses: Learning to Retrieve Dance Poses from Music
Tseng, B.-W., Yang, K. C. C., Hu, Y.-H., Wei, W.-L., & Lin, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446425 · Cited by 1
“In contrast to existing methods [2, 3, 4, 5, 6, 7] that treat this task as a generation problem and only produce very limited or even singular dance sequence variations from music, our method, by treating it as a retrieval challenge, offers a broader spectrum of possibilities for choreographers,…”
BackgroundMethodInfluentialControlling Character Motions without Observable Driving Source
Li, W., Dai, B., Zhou, Z., Yao, Q., & Wang, B. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00608 · Cited by 2
Embodied Human Activity Recognition
Hu, S., Gong, Y., & Mori, G. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00632 · Cited by 1
“To simulate an embodied agent which can move and perceive realistic human activities in 3D space, we use real motion capture data from the Extreme Pose Interaction (ExPI) dataset [19] and the AIST++ dataset [33].”
MethodTMMM: Transformer in Multimodal Sentiment Analysis under Missing Modalities
Wu, H., Li, S., & Wu, D. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651217 · Cited by 1
“In the second half, tokens from multiple modalities are concatenated and fed into a shared transformer block for fusion and self-attention [14].”
BackgroundEra, Y., Togo, R., Maeda, K., Ogawa, T., & Haseyama, M. (2024) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan62264.2024.10674583 · Cited by 1
“We used the dance skeletal data in the Aist++ dataset [3] as m d and the labeled skeletal data in the dataset based on Xia et al. [4] as m s , and defined the 95% of m d and m s as the train data and the remaining as the test data.”
MethodDanceCaps: Pseudo-Captioning for Dance Videos Using Large Language Models
Kim, S., & Lee, K. (2024) · Applied Sciences · doi:10.3390/app142210116 · Cited by 1
MDRPC: Music-Driven Robot Primitives Choreography
Guan, H., Wei, X., Long, W., Yang, D., Zhai, P., & Zhang, L. (2024) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai62512.2024.00111 · Cited by 1
“The AI Choreographer [7] proposed the full attention cross-modal transformer model to learn the correlation between music and motion, using seed motion and music segments to auto-regressively generate long-range future motion.”
Metadata
- Authors
- Li, R., Yang, S., Ross, D. A., & Kanazawa, A.
- Affiliations
- University of Southern California (US)
- Google (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13381-13392
- OpenAlex
- W3204221554
- Cited by
- 553
- References
- 94
- Citations by year
- 2026: 92 · 2025: 146 · 2024: 154 · 2023: 113 · 2022: 46 · 2021: 2
- FWCI
- 117.49
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- cross-modal transformer, full attention, AIST, dance genres, choreography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
