The DOI matches its Crossref record; 90 of the 94 references it cites were found in a registry or an index.
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
Open accessAbstract
We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music. The proposed AIST++ dataset contains 5.2 hours of 3D dance motion in 1408 sequences, covering 10 dance genres with multi-view videos with known camera poses—the largest dataset of this kind to our knowledge. We show that naively applying sequence models such as transformers to this dataset for the task of music conditioned 3D motion generation does not produce satisfactory 3D motion that is well correlated with the input music. We overcome these shortcomings by introducing key changes in its architecture design and supervision: FACT model involves a deep cross-modal transformer block with full-attention that is trained to predict N future motions. We empirically show that these changes are key factors in generating long sequences of realistic dance motion that are well-attuned to the input music. We conduct extensive experiments on AIST++ with user studies, where our method outperforms recent state-of-the-art methods both qualitatively and quantitatively. The code and the dataset can be found at: https://google.github.io/aichoreographer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We note that the simple prediction-based late fusion [247], [248] is less adopted in MML Transformers.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [248]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 8×
“We propose Full Attention Cross-Modal Transformer (FACT) model for the task of 3D dance motion generation.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“All our experiments are trained with 16 batch size using Adam [45] optimizer.”
MethodJakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“On the other hand, models like BERT [17] employ full-attention for feature learning, but rarely are these models employed in an auto-regressive manner, due to its inefficiency at test time.”
MethodGANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 1×
“Training data for 2D pose and audio is abundant thanks to the high reliability of 2D pose detectors [14].”
BackgroundDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 2×
“While there are many large scale 3D motion capture datasets [39, 59, 1, 37], mocap dataset of 3D dance is quite limited as it requires heavy instrumentation and expert dancers for capture.”
BackgroundMethod
Research works printed but not linked: 3
- lamothe, 2019, The dancing species: how moving together in time helps make us human, Aeon
- alemi, 2017, Groovenet: Real-time music-driven dance movement generation using artificial neural networks, Networks, 8, 26
- chen, 2021, Choreomaster: choreography-oriented music-driven dance synthesis, ACM Transactions on Graphics (TOG), 40, 1
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
“Depth to Binaural Audio [14], music piece & seed 3D motion to long-range future 3D motions [146], X-raying image & question to answer [244], video & text & audio to text [245]), and”
BackgroundAction-Conditioned 3D Human Motion Synthesis with Transformer VAE
Petrovich, M., Black, M. J., & Varol, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01080 · Cited by 571
“Very recently, [39] presents a Transformer-based method for dance generation conditioned on music and past motion.”
MethodMotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355414 · Cited by 890
“, 2018), 3D keypoints (Ionescu et al., 2013; Joo et al., 2015; Mehta et al., 2017; Trumble et al., 2017; Li et al., 2021) and statistical model parameters (Yu et al.”
BackgroundMotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437
“, 2018), 3D keypoints (Ionescu et al., 2013; Joo et al., 2015; Mehta et al., 2017; Trumble et al., 2017; Li et al., 2021) and statistical model parameters (Yu et al.”
BackgroundTEMOS: Generating Diverse Human Motions from Textual Descriptions
Petrovich, M., Black, M. J., & Varol, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20047-2_28 · Cited by 429
Executing your Commands via Motion Diffusion in Latent Space
Chen, X., Jiang, B., Liu, W., Huang, Z., Fu, B., Chen, T., & Yu, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01726 · Cited by 452
“Human Motion Synthesis allows rich inputs of multimodal data, such as text [16, 47, 68, 18, 2, 28], action category [46, 19], incomplete pose sequences [12, 20, 68], control signals [65, 66, 45], musics [34, 33, 32] and image(s) [55, 9], here, we focus on some typical tasks.”
BackgroundMotionCLIP: Exposing Human Motion Generation to CLIP Space
Tevet, G., Gordon, B., Hertz, A., Bermano, A. H., & Cohen‐Or, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20047-2_21 · Cited by 397
Generating Human Motion from Textual Descriptions with Discrete Representations
Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., & Ying, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01415 · Cited by 328
“Motion can also be generated with music to produce 3D dance motion [6,13,36,37,39,40].”
BackgroundLi, Y., El Habib Daho, M., Conze, P.-H., Zeghlache, R., Le Boité, H., Tadayoni, R., Cochener, B., Lamard, M., & Quellec, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108635 · Cited by 274
“…Transformer-based multi-modal fusion networks into self-attention Transformers (Akbari et al., 2021; Nagrani et al., 2021; Shi et al., 2022; Li et al., 2021; Pashevich et al., 2021; Appalaraju et al., 2021; Steitz et al., 2022; Wu and Mebane Jr, 2022) and cross-attention Trans-formers (Lu…”
BackgroundEDGE: Editable Dance Generation From Music
Tseng, J., Castellon, R., & Liu, C. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00051 · Cited by 332
“Dataset In this work, we use AIST++ [30], a dataset consisting of 1,408 high-quality dance motions paired to music from a diverse set of genres.”
BackgroundMethodResultInfluentialPhysDiff: Physics-Guided Human Motion Diffusion Model
Yuan, Y., Song, J., Iqbal, U., Vahdat, A., & Kautz, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01467 · Cited by 311
“These methods often use GANs or VAEs to generate motions from various conditions such as past motions [3, 7, 87, 91, 92], key frames [24], music [37, 38, 100], text [1, 9, 22, 62], and action labels [10, 23, 61].”
MethodHong, F., Zhang, M., Pan, L., Cai, Z., Yang, L., & Liu, Z. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530094 · Cited by 335
“Some other works [Aggarwal and Parikh 2021; Li et al. 2021] focus on music-conditioned motion synthesis.”
BackgroundBailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
Li, S., Yu, W., Gu, T., Lin, C., Wang, Q., Qian, C., Loy, C. C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01077 · Cited by 208
“To further understand the real visual performance of our method, we conduct a user study among the dance sequences generated by each compared method and the ground truth data in AIST++ test set.”
BackgroundMethodInfluentialListen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Alexanderson, S., Nagy, R., Beskow, J., & Henter, G. E. (2023) · ACM Transactions on Graphics · doi:10.1145/3592458 · Cited by 206
“…combinations of GANs and VAEs [Lee et al. 2019b], dilated convolutions and gated activation units [Zhuang et al. 2022], Transformer architectures [Li et al. 2022, 2020, 2021; Valle-Pérez et al. 2021], VQ-VAEs and Transformers with re-inforcement learning [Siyao et al. 2022], and Conformers…”
Background3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting
Qian, Z., Wang, S., Mihajlovic, M., Geiger, A., & Tang, S. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.00480 · Cited by 279
“We observe that our method preserves sharper details compared to ARAH and does not produce fluctuating artifacts as in HumanNeRF caused by noisy de-Table Additionally, we animate our learned avatars with pose sequences from AMASS [27] and AIST++ [20], shown in the rightmost column of Fig.”
MethodMoFusion: A Framework for Denoising-Diffusion-Based Motion Synthesis
Dabral, R., Mughal, M. H., Golyanik, V., & Theobalt, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00941 · Cited by 202
“Moreover, earlier methods [28,29,62] use hand-crafted music features (such as beats, chroma and onset strength) along with MFCC representation of audio signals for predicting music-aligned dance sequences.”
InfluentialGuided Motion Diffusion for Controllable Human Motion Synthesis
Karunratanakul, K., Preechakul, K., Suwajanakorn, S., & Tang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00205 · Cited by 218
“Various conditions have been explored such as partial poses [14, 18, 54], trajectories [25, 56, 63], images [9,46], music [29,30,32], text [1,15,16,26,40], objects [59], action labels [17,39], or unconditioned [37,60,64,66].”
BackgroundBEAT: A Large-Scale Semantic and Emotional Multi-modal Dataset for Conversational Gestures Synthesis
Liu, H., Zhu, Z., Iwamoto, N., Peng, Y., Li, Z., Zhou, Y., Bozkurt, E., & Zheng, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_36 · Cited by 164
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ruan, L., Ma, Y., Yang, H., He, H., Liu, B., Fu, J., Yuan, N. J., Jin, Q., & Guo, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00985 · Cited by 194
“Even in AIST++, almost half of the generated sound videos can fool users although the fine-grained parts of persons are difficult to be well generated.”
BackgroundMethodInfluentialDanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer
Li, B., Zhao, Y., Shi, Z., & Sheng, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20014 · Cited by 143
“We follow the official training/testing splits in AIST++ (Li et al. 2021).”
BackgroundMethodResultInfluentialMultimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 197
BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion
Black, M. J., Patel, P., Tesch, J., & Yang, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00843 · Cited by 148
“[41] Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa.”
BackgroundTEACH: Temporal Action Composition for 3D Humans
Athanasiou, N., Petrovich, M., Black, M. J., & Varol, G. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00053 · Cited by 168
“More relevant here is work that generates human motion conditioned on language [1, 2, 13, 30, 31] or music [45, 21].”
BackgroundGLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Yuan, Y., Iqbal, U., Molchanov, P. A., Kitani, K., & Kautz, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01076 · Cited by 141
“(4) FID , which is an extension of the original Frechet Inception Distance that calculates the distribution distance between estimated motions and the GT. FID is a standard metric in motion generation literature to evaluate the quality of generated motions [34,58,59,98].”
BackgroundMethodTaming Diffusion Models for Audio-Driven Co-Speech Gesture Generation
Zhu, L., Liu, X., Liu, X., Qian, R., Liu, Z., & Yu, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01016 · Cited by 152
“Proposed in [24, 26], BC measures motion-audio beat correlation.”
BackgroundState of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 198
Self-citationAo, T., Gao, Q., Lou, Y., Chen, B., & Liu, L. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555435 · Cited by 126
“…studies use deep encoders to extract audio features from raw audio signals or audio spectrograms, where only the features extracted at the last layer of the encoder are used to generate gestures [Alexanderson et al. 2020; Kucherenko et al. 2020; Li et al. 2021a; Qian et al. 2021; Yoon et al. 2020].”
BackgroundMethodInfluentialInterGen: Diffusion-Based Multi-human Motion Generation Under Complex Interactions
Liang, H., Zhang, W., Li, W., Jingyi Yu, A., & Xu, L. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02042-6 · Cited by 172
“…motion sequences (Duan et al., 2021; Harvey et al., 2020), control signals (Starke et al., 2022; Peng et al., 2021; Starke et al., 2019), music (Li et al., 2021, 2022; Lee et al., 2019), speech (Habibie et al., 2022; Ao et al., 2022), scene (Wang et al., 2021, 2022) and images (Rempe et al.,…”
InfluentialGOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping
Taheri, O., Choutas, V., Black, M. J., & Tzionas, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01291 · Cited by 136
“More recent methods [36, 41, 49, 58] use attention [59].”
DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection
Tao, Y., Yang, M., Li, H., Wu, Y., & Hu, B. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3350071 · Cited by 119
“…based on the Transformer and Attention mechanisms in various fields has contributed significantly to the development of their respective fields, e.g. humour detection [35], product detection [36], data alignment in time series modelling of human behaviour [37], dance choreography [38], and so on.”
BackgroundHuman Motion Generation: A Survey
Zhu, W., Ma, X., Ro, D., Ci, H., Zhang, J., Shi, J., Gao, F., Tian, Q. C., & Wang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3330935 · Cited by 141
“AIST++ [115] is constructed from the AIST Dance Video DB [189].”
BackgroundMethodInfluentialSmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos
Zeng, A., Yang, L., Ju, X., Li, J., Wang, J., & Xu, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_36 · Cited by 115
HumanMAC: Masked Motion Completion for Human Motion Prediction
Chen, L.-H., Zhang, J., Li, Y., Pang, Y., Xia, X., & Liu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00875 · Cited by 119
“UDE [85] and MoFusion [12] propose to unify the audio-/text-driven motion generation task [33, 50, 25, 62, 58, 71, 49, 51, 31] in one system.”
BackgroundARAH: Animatable Volume Rendering of Articulated Human SDFs
Wang, S., Schwarz, K., Geiger, A., & Tang, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_1 · Cited by 112
HuMMan: Multi-modal 4D Human Dataset for Versatile Sensing and Modeling
Cai, Z., Ren, D., Zeng, A., Lin, Z., Yu, T., Wang, W., Fan, X., Gao, Y., Yu, Y., Pan, L., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_33 · Cited by 110
Liu, H., Zhu, Z., Becherini, G., Peng, Y., Su, M., Zhou, Y., Zhe, X., Iwamoto, N., Zheng, B., & Black, M. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00115 · Cited by 145
“Then, we measure Diversity [35] by calculating the average L1 distance between multiple body gesture clips, and use BC [37] to assess speech-motion synchrony.”
BackgroundMethodCOUCH: Towards Controllable Human-Chair Interactions
Zhang, X., Bhatnagar, B. L., Starke, S., Guzov, V., & Pons‐Moll, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_30 · Cited by 87
Disco: Disentangled Control for Realistic Human Dance Generation
Wang, T., Li, L., Lin, K., Zhai, Y., Lin, C.-C., Yang, Z., Zhang, H., Liu, Z., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00891 · Cited by 174
“In addition to the TikTok data, we show strong zero-shot generation results of D IS C O on several unseen out-of-distribution (OOD) data: Everybody Dance Now [5], AIST++ [31] and even wild youtube video in Figure 7.”
BackgroundPiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection
Chen, A., Zhang, K., Zhang, R., Wang, Z., Lu, Y., Guo, Y., & Zhang, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00512 · Cited by 97
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos
Liu, S., Tripathi, S., Majumdar, S., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00328 · Cited by 82
“Transformers have also been widely used in video forecasting problems like action anticipation [28], trajectory estimation [30,90] and human motion prediction [45,66].”
MethodFineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation
Li, R., Zhao, J., Zhang, Y., Su, M., Ren, Z., Zhang, H., Tang, Y., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00939 · Cited by 95
“Currently, the largest available dataset is AIST++[24], which is collected by reconstructing 3D poses in multi-view videos.”
BackgroundMethodInfluentialSelf-citationMotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Zhang, Y., Huang, D., Liu, B., Tang, S., Lu, Y., Chen, L., Bai, L., Chu, Q., Yu, N., & Ouyang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28567 · Cited by 128
Rhythm is a Dancer: Music-Driven Motion Synthesis With Global Structure
Aristidou, A., Yiannakidis, A., Aberman, K., Cohen‐Or, D., Shamir, A., & Chrysanthou, Y. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3163676 · Cited by 73
“Recently, new approaches are focused on adding motion diversity into the synthesis process [79], [80], and try to use of more sophisticated architectures, such as transformers [80], [81].”
BackgroundMethodDNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Cheng, W., Chen, R., Fan, S., Yin, W., Chen, K., Cai, Z., Wang, J., Gao, Y., Yu, Z., Lin, Z., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01829 · Cited by 91
“AIST++ [56, 28] is a dance database with various dance motions while sticking in the one-fold scenario and lacking view density.”
BackgroundTM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration
Gong, K., Lian, D., Chang, H., Guo, C., Jiang, Z., Zuo, X., Mi, M. B., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00912 · Cited by 68
“There exist two significant challenges to be considered: i) the existing datasets only cater to either music-driven (music2dance) [42, 5, 51, 27] or text-driven (text2motion) [34, 15] human motion generation, and no paired 3D dance generation dataset exists that takes into account both music and text.”
BackgroundMethodInfluentialLi, R., Zhang, Y.-X., Zhang, Y., Zhang, H., Guo, J., Zhang, Y. T., Liu, Y., & Li, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00151 · Cited by 77
“FACT [22] and MNET [19] are auto-gressive dance generation methods.”
BackgroundMethodInfluentialSelf-citationGIMO: Gaze-Informed Human Motion Prediction in Context
Zheng, Y., Yang, Y., Mo, K., Li, J., Yu, C., Liu, Y., Liu, C. K., & Guibas, L. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_39 · Cited by 75
Implicit Neural Representations for Variable Length Human Motion Generation
Cervantes, P., Sekikawa, Y., Sato, I., & Shinoda, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_22 · Cited by 67
A Brand New Dance Partner: Music-Conditioned Pluralistic Dancing Controlled by Multiple Dance Genres
Kim, J., Oh, H., Kim, S., Tong, H., & Lee, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00348 · Cited by 66
“FACT [38] present a full-attention based crossmodal transformer that adopts sequence-to-sequence learning to generate more realistic 3D dance sequence.”
BackgroundMethodInfluentialMultimodal fusion models for pulmonary embolism mortality prediction
Cahan, N., Klang, E., Marom, E. M., Soffer, S., Barash, Y., Burshtein, E., Konen, E., & Greenspan, H. K. (2023) · Scientific Reports · doi:10.1038/s41598-023-34303-8 · Cited by 63
MotionLCM: Real-Time Controllable Motion Generation via Latent Consistency Model
Dai, W., Chen, L.-H., Wang, J., Liu, J., Dai, B., & Tang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_22 · Cited by 61
UDE: A Unified Driving Engine for Human Motion Generation
Zhou, Z., & Wang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00545 · Cited by 69
“2) For the audio-to-motion task, ours also shows the highest quality and correlation compared with [18] and [28].”
BackgroundMethodResultInfluentialChen, J., Liu, Y., Wang, J., Zeng, A., Li, Y., & Chen, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00702 · Cited by 70
“… such a true story … Prior to the audio input’s emphasis on the word “five”, our approach instinctively raises the head to prepare for highlighting, subsequently producing precise lip movements accompanied by raised eyebrows to emphasize the word “five”. method shows more agile gestures than CaMN.”
BackgroundLiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors
Ren, Y., Zhao, C., He, Y., Cong, P., Liang, H., Jingyi Yu, A., Xu, L., & Ma, Y. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3247088 · Cited by 52
“We conduct extensive experiments on LIPD and a variety of other real and synthetic datasets [19,32,33,38] to demonstrate the capability of our method LIP for capturing challenging human motions in various large-scale scenes.”
MethodLiu, H., Iwamoto, N., Zhu, Z., Li, Z., Zhou, Y., Bozkurt, E., & Zheng, B. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548400 · Cited by 55
“In this paper, similar as [48] we take the RMS onset of speech as the potential speech audio beat and calculate the speech-motion synchrony by Align.”
BackgroundMethodInfluentialSINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation
Athanasiou, N., Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00916 · Cited by 62
“Example conditions include music [32, 40], speech [1, 17], scenes [20, 53, 59, 69], action [16, 43] or text [2, 4, 13, 15, 33, 44].”
BackgroundMake-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation
Azadi, S., Shah, A., Hayes, T., Parikh, D., & Gupta, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01381 · Cited by 54
“Human pose and motion generation Prior works in human pose and motion synthesis have explored generative models either unconditionally [36, 40, 25], or conditioned on various input signals such as a prior motion [20, 29], an action class [11, 23, 5], or music [17, 18].”
BackgroundUnsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations
Noguchi, A., Sun, X. N., Lin, S., & Harada, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_36 · Cited by 58
Priority-Centric Human Motion Generation in Discrete Latent Space
Kong, H., Gong, K., Lian, D., Mi, M. B., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01360 · Cited by 56
“Conditional human motion synthesis Compared with unconditional Human motion synthesis, human motion synthesis conditioned on various conditions, such as textual description [55, 17, 5, 56], music [27, 28, 29, 13], and actions [34, 18] is more challenging due to the huge gap between two different domains.”
BackgroundFg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model
Wang, Y., Leng, Z., Li, F. W. B., Wu, S., & Liang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02014 · Cited by 56
“The input multimodal data include music [12, 15, 25, 27], motion cat-”
BackgroundMusic-Driven Group Choreography
Le, N., Pham, T., Do, T., Tjiputra, E., Tran, Q. D., & Nguyen, A. Q. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00838 · Cited by 53
“Currently, most of the music-to-dance datasets [37, 39, 47, 55, 58, 66] only contain solo dance videos.”
BackgroundMethodInfluentialHumanBench: Towards General Human-Centric Perception with Projector Assisted Pretraining
Tang, S., Chen, C., Xie, Q., Chen, M., Wang, Y., Ci, Y., Bai, L., Zhu, F., Yang, H., Li, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02104 · Cited by 48
“6M [10] 312,187 Paper Doll [35] 1,035,825 AIST++ [13] 1,015,257 DeepFashion [21] 191,961”
Quantized GAN for Complex Music Generation from Dance Videos
Zhu, Y., Olszewski, K., Wu, Y., Achlioptas, P., Chai, M., Yan, Y., & Tulyakov, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_11 · Cited by 44
Vision + X: A Survey on Multimodal Learning in the Light of Data
Zhu, Y., Wu, Y., Sebe, N., & Yan, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3420239 · Cited by 56
“Additionally, AIST++ [138] is a subset of AIST with more refined dance motion annotations.”
BackgroundMethodLearning Locally Editable Virtual Humans
Ho, H.-I., Xue, L., Song, J., & Hilliges, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02014 · Cited by 55
“With the advances in neural rendering [63,64] and the availability of large-scale human datasets [18,28,29,32,47,67,73,74], numerous approaches have been proposed to reconstruct [4,56,57] and explicitly control [10,30,50] human avatars in a data-driven manner.”
BackgroundEMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation
Zhou, W., Dou, Z., Cao, Z., Liao, Z., Wang, J., Wang, W., Liu, Y., Komura, T., Wang, W., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72627-9_2 · Cited by 71
Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation
Petrovich, M., Litany, O., Iqbal, U., Black, M. J., Varol, G., Peng, X. B., & Rempe, D. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00197 · Cited by 64
“Generating motions from text descriptions is of particular interest; it has the potential to democratize animation with a natural language interface that is intuitive for beginner and expert users alike.”
BackgroundThe Wanderings of Odysseus in 3D Scenes
Zhang, Y., & Tang, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01983 · Cited by 45
CaV3: Cache-assisted Viewport Adaptive Volumetric Video Streaming
Liu, J., Zhu, B., Wang, F., Jin, Y., Zhang, W., Xu, Z., & Cui, S. R. (2023) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr55154.2023.00033 · Cited by 40
Gaussian Shell Maps for Efficient 3D Human Generation
Abdal, R., Wang, Y., Shi, Z., Xu, Y., Po, R., Kuang, Z., Chen, Q., Yeung, D., & Wetzstein, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00902 · Cited by 57
Zheng, J., Zheng, Q., Fang, L., Liu, Y., & Yi, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00064 · Cited by 38
Optimizing Diffusion Noise Can Serve As Universal Motion Priors
Karunratanakul, K., Preechakul, K., Aksan, E., Beeler, T., Suwajanakorn, S., & Tang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00133 · Cited by 69
“Various conditioning signals have been explored such as partial poses [15, 20], trajectories [33, 65, 76, 87], images [8, 58], music [37, 38, 41], objects [78], action labels [16, 51], scene [27], or text [1, 18, 19, 34, 52].”
BackgroundSárándi, I., Hermans, A., & Leibe, B. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00297 · Cited by 54
“81M >60 19 ♣♡ COCO AIST-Dance++ [45, 75] 1.”
An Empirical Study on How People Perceive AI-generated Music
Chu, H., Kim, J., Kim, S., Lim, H., Lee, H., Jin, S., Lee, J., Kim, T.-H., & Ko, S. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557235 · Cited by 39
“Recently, several studies propose methods link visual and auditory experience: generating natural sound for in-the-wild videos [76], generating music from musical instrument playing video [20], or generating 3D dance movement from music [36].”
BackgroundInteraction Transformer for Human Reaction Generation
Chopin, B., Tang, H., Otberdout, N., Daoudi, M., & Sebe, N. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3242152 · Cited by 49
“Inspired by the recent advances in generative models, several works [8]–[11] proposed human action generation models in order to generate a consecutive sequence of human motions.”
BackgroundEmotional Speech-Driven 3D Body Animation via Disentangled Latent Diffusion
Chhatre, K., Daněček, R., Athanasiou, N., Becherini, G., Peters, C., Black, M. J., & Bolkart, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00190 · Cited by 52
“The scores are obtained from the BEAT authors, representing a continuous score on a scale 0-1 per gesture style for 4 gesture semantic categories: beat, deictic, iconic, and metaphoric.”
MethodVideo Background Music Generation: Dataset, Method and Evaluation
Zhuo, L., Wang, Z., Wang, B., Liao, Y., Bao, C., Peng, S., Han, S., Zhang, A., Fang, F., & Liu, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01433 · Cited by 36
“Multi-modal datasets, e.g. , image-text [41, 6, 42], video-text [33, 55], video-audio [7, 15, 30, 32] greatly push the development of multi-modal learning tasks.”
BackgroundPoseFix: Correcting 3D Human Poses with Natural Language
Delmas, G., Weinzaepfel, P., Moreno-Noguer, F., & Rogez, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01379 · Cited by 40
“Previous works have mainly focused on the generation of pose sequences, conditioning on music [28, 29], context [10], past poses [61, 62], text labels [20, 44] and mostly on text descriptions [30, 1, 59, 2, 17, 45, 18, 54, 19, 26].”
BackgroundTLControl: Trajectory and Language Control for Human Motion Synthesis
Wan, W., Dou, Z., Komura, T., Wang, W., Jayaraman, D., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_3 · Cited by 45
Evaluating Gesture Generation in a Large-scale Open Challenge: The GENEA Challenge 2022
Kucherenko, T., Wolfert, P., Yoon, Y., Viegas, C., Nikolov, T., Tsakov, M., & Henter, G. E. (2024) · ACM Transactions on Graphics · doi:10.1145/3656374 · Cited by 49
“Beat consistency, which was first proposed for dance motion [Li et al. 2021], has also been used to assess gesture generation [Liu et al. 2022a].”
MethodRole-aware Interaction Generation from Textual Description
Tanaka, M., & Fujiwara, K. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01466 · Cited by 50
“Various approaches have been proposed to learn to generate motion from data, including methods for generating appropriate actions given action labels [10, 36, 5], research on generating actions that match music [24, 25], and methods that focus on the periodicity of actions [48].”
MethodEgoHumans: An Egocentric 3D Multi-Human Benchmark
Khirodkar, R., Bansal, A., Ma, L., Newcombe, R., Vo, M., & Kitani, K. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01814 · Cited by 44
“6M [43], AMASS [78], HumanEva [103], AIST++ [66], HUMBI [119], PROX [37], and TotalCap-ture [49] only contain single human sequences.”
BackgroundInfluentialBioinspired In-Sensor Multimodal Fusion for Enhanced Spatial and Spatiotemporal Association
Ma, S., Zhou, Y., Wan, T., Ren, Q., Yan, J., Fan, L., Yuan, H., Chan, M., & Chai, Y. (2024) · Nano Letters · doi:10.1021/acs.nanolett.4c01727 · Cited by 35
Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation
Wang, Y., Mu, L., Liu, J., Leng, Z., Li, F. W. B., Zhang, Z., & Liang, X. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02392-9 · Cited by 44
“While existing works utilize diverse multimodal inputs such as music (Kao & Su, 2020; Li et al., 2021; Ren et al., 2020; Starke et al., 2022; Tseng et al., 2023), motion categories (Guo et al., 2020; Petrovich et al., 2021; Cervantes et al., 2022; Guo et al., 2022c) and trajectories (Karunratanakul…”
IDOL: Instant Photorealistic 3D Human Creation from a Single Image
Zhuang, Y., Lv, J., Wen, H., Shuai, Q., Zeng, A., Zhu, H., Chen, S., Yang, Y., Cao, X., & Liu, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02450 · Cited by 57
“X)”
Seamless Human Motion Composition with Blended Positional Encodings
Barquero, G., Escalera, S., & Palmero, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00051 · Cited by 44
“…motion generation have shown notable progress in synthesizing movements conditioned on diverse modalities such as text [21, 26, 27, 35, 40, 61, 78, 79, 97–99], music [2, 17, 46, 74, 81, 93, 107], scenes [15, 30, 84–86, 94 ], interactive interactive objects Autoregressive human motion composition.”
BackgroundLivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation
Zhi, Y., Cun, X., Chen, X., Shen, X., Guo, W. Q., Huang, S., & Gao, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01902 · Cited by 44
“[37] generate the dance motions from a transformer-based network and a high-quality dataset.”
BackgroundDanceConv: Dance Motion Generation with Convolutional Networks
Kritsis, K., Gkiokas, A., Pikrakis, A., & Katsouros, V. (2022) · IEEE Access · doi:10.1109/access.2022.3169782 · Cited by 35
“…of the predicted skeletal poses and computed various diversity metrics which are usually employed in related studies [13], [20], [25], [56], [57], including: 1) The Fréchet Inception Distance (FID) [67] score that measures the distance between the distributions of the generated poses and…”
MethodBailando ++: 3D Dance GPT With Choreographic Memory
Li, S., Yu, W., Gu, T., Lin, C., Wang, Q., Qian, C., Loy, C. C., & Liu, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3319435 · Cited by 36
“Unless specifically mentioned, we use the rotation matrix to represent the 3D joint angles as in [7].”
BackgroundMethodResultInfluentialAu, H.-Y., Chen, J., Jiang, J., & Guo, Y.-K. (2022) · ACM Multimedia · doi:10.1145/3503161.3547797 · Cited by 42
“becomes the hot trend of research [17, 18, 29, 32].”
BackgroundMethodInfluentialPerspectives on Advancing Multimodal Learning in Environmental Science and Engineering Studies
Liu, W., Chen, J., Wang, H., Fu, Z., Peijnenburg, W. J. G. M., & Hong, H. (2024) · Environmental Science & Technology · doi:10.1021/acs.est.4c03088 · Cited by 34
DanceGen: Supporting Choreography Ideation and Prototyping with Generative AI
Liu, Y., & Sra, M. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661594 · Cited by 38
“To generate diverse dance sequences based on text, we employed a pre-trained text-to-motion generation model, MDM [74], as the backbone and fine-tuned it on a large-scale dance dataset AIST++ [45].”
BackgroundMethodInfluentialA Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision
Zhao, Z., Chai, W., Hao, S., Hu, W., Wang, G., Cao, S., Song, M., Hwang, J., & Wang, G. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3554801 · Cited by 56
“EA-MUD has 104 videos of 12 genres, while AIST++ is a massive 3D human dance motion dataset with annotations, including camera parameters, human joint locations, and pose parameters.”
InfluentialWith a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition
Kazakos, E., Huh, J., Nagrani, A., Zisserman, A., & Damen, D. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.171 · Cited by 43
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 32
“Later studies approached this task by employing music [69] and text [35], [62], [70] as input conditions to construct versatile and realistic human motion.”
MethodDeciWatch: A Simple Baseline for $$10\times $$ Efficient 2D and 3D Pose Estimation
Zeng, A., Ju, X., Yang, L., Gao, R., Zhu, X., Dai, B., & Xu, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_35 · Cited by 29
DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
Qi, Q., Zhuo, L., Zhang, A., Liao, Y., Fang, F., Liu, S., & Yan, S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612307 · Cited by 38
“Recently, AIST++ [21], a large-scale 3D motion dataset, greatly pushed the development of 3D dance generation.”
BackgroundMethodInfluentialActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs
Mu, J., Sang, S., Vasconcelos, N. M., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01686 · Cited by 28
“We quantitatively and qualitatively demonstrate that ActorsNeRF outperforms the existing approaches by a large margin on various few-shot settings for both ZJU-MoCap Dataset [33] and AIST++ Dataset [21].”
BackgroundMethodInfluentialA Unified Framework for Real Time Motion Completion
Duan, Y., Lin, Y., Zou, Z., Yuan, Y. G., Qian, Z., & Zhang, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20368 · Cited by 29
“…“free dance”, the Anidance dataset (Tang, Mao, and Jia 2018) cannot be directly applied in the game environment since the rotation of the joints are not provided, and AIST++ dataset is built by fitting methods rather than Mocap devices (Loper et al. 2015; Kolotouros et al. 2019; Li et al. 2021).”
MethodDance with You: The Diversity Controllable Dancer Generation via Diffusion Models
Yao, S., Sun, M., Li, B., Yang, F., Wang, J., & Zhang, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612046 · Cited by 31
“Furthermore, the classifier-free guidance [15] is proposed to avoid training an extra classification network.”
Method
Metadata
- Authors
- Li, R., Yang, S., Ross, D. A., & Kanazawa, A.
- Affiliations
- University of Southern California (US)
- Google (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13381-13392
- OpenAlex
- W3204221554
- Cited by
- 553
- References
- 94
- Citations by year
- 2026: 92 · 2025: 146 · 2024: 154 · 2023: 113 · 2022: 46 · 2021: 2
- FWCI
- 117.49
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- cross-modal transformer, full attention, AIST, dance genres, choreography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
