AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
Open accessAbstract
We present AIST++, a new multi-modal dataset of 3D dance motion and music, along with FACT, a Full-Attention Cross-modal Transformer network for generating 3D dance motion conditioned on music. The proposed AIST++ dataset contains 5.2 hours of 3D dance motion in 1408 sequences, covering 10 dance genres with multi-view videos with known camera poses—the largest dataset of this kind to our knowledge. We show that naively applying sequence models such as transformers to this dataset for the task of music conditioned 3D motion generation does not produce satisfactory 3D motion that is well correlated with the input music. We overcome these shortcomings by introducing key changes in its architecture design and supervision: FACT model involves a deep cross-modal transformer block with full-attention that is trained to predict N future motions. We empirically show that these changes are key factors in generating long sequences of realistic dance motion that are well-attuned to the input music. We conduct extensive experiments on AIST++ with user studies, where our method outperforms recent state-of-the-art methods both qualitatively and quantitatively. The code and the dataset can be found at: https://google.github.io/aichoreographer.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Simple prediction-based late fusion is less adopted in MML Transformers.
“We note that the simple prediction-based late fusion [247], [248] is less adopted in MML Transformers.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [248]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734 · see citing works
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · see citing works
librosa: Audio and Music Signal Analysis in Python
McFee, B., Raffel, C., Liang, D., Ellis, D. P. W., McVicar, M., Battenberg, E., & Nieto, O. (2015) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-7b98e3ed-003 · Cited by 3,099 · see citing works
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.471 · Cited by 2,699 · see citing works
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621 · see citing works
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2015.515 · Cited by 1,507 · see citing works
AMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323 · see citing works
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,279 · see citing works
VideoBERT: A Joint Model for Video and Language Representation Learning
Sun, C., Myers, A., Vondrick, C., Murphy, K. P., & Schmid, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00756 · Cited by 1,212 · see citing works
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133 · see citing works
Kovar, L., Gleicher, M. L., & Pighin, F. P. (2008) · ACM SIGGRAPH Courses · doi:10.1145/1401132.1401202 · Cited by 1,073 · see citing works
VIBE: Video Inference for Human Body Pose and Shape Estimation
Kocabas, M., Athanasiou, N., & Black, M. J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00530 · Cited by 1,065 · see citing works
Describing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035 · see citing works
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R. J., & Saenko, K. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1173 · Cited by 996 · see citing works
Recurrent Network Models for Human Dynamics
Fragkiadaki, K., Levine, S., Felsen, P., & Malik, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.494 · Cited by 904 · see citing works
A Comparative Study on Transformer vs RNN in Speech Applications
Karita, S., Chen, N., Hayashi, T., Hori, T., Inaguma, H., Jiang, Z., Someki, M., Yalta, N., Yamamoto, R., Wang, X., et al. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003750 · Cited by 766 · see citing works
FastSpeech: Fast, Robust and Controllable Text to SpeechPreprint
Ren, Y., Ruan, Y., Tan, X., Qin, T., Zhao, S., Zhao, Z., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09263 · Cited by 724 · see citing works
Brand, M., & Hertzmann, A. (2000) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/344779.344865 · Cited by 704 · see citing works
A deep learning framework for character motion synthesis and editing
Holden, D., Saito, J., & Komura, T. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925975 · Cited by 656 · see citing works
End-to-End Dense Video Captioning with Masked Transformer
Zhou, L., Zhou, Y., Corso, J. J., Socher, R., & Xiong, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00911 · Cited by 605 · see citing works
A hierarchical approach to interactive motion editing for human-like figures
Lee, J., & Shin, S. Y. (1999) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/311535.311539 · Cited by 569 · see citing works
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
Yu, H., Wang, J., Huang, Z., Yang, Y., & Xu, W. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.496 · Cited by 568 · see citing works
Phase-functioned neural networks for character control
Holden, D., Komura, T., & Saito, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073663 · Cited by 556 · see citing works
LPCNET: Improving Neural Speech Synthesis through Linear Prediction
Valin, J.-M., & Skoglund, J. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682804 · Cited by 427 · see citing works
Deep Representation Learning for Human Motion Prediction and Classification
Bütepage, J., Black, M. J., Kragić, D., & Kjellström, H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.173 · Cited by 414 · see citing works
Lu, J., Yang, J., Batra, D., & Parikh, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00754 · Cited by 412 · see citing works
Entangled Transformer for Image Captioning
Li, G., Zhu, L., Liu, P., & Yang, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00902 · Cited by 406 · see citing works
Interactive motion generation from examples
Arıkan, O., & Forsyth, D. A. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566606 · Cited by 391 · see citing works
Factored conditional restricted Boltzmann Machines for modeling motion style
Taylor, G. W., & Hinton, G. E. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553505 · Cited by 362 · see citing works
Huang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275108 · Cited by 342 · see citing works
Learning Individual Styles of Conversational Gesture
Ginosar, S., Bar, A., Kohavi, G., Chan, C., Owens, A., & Malik, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00361 · Cited by 341 · see citing works
Neural state machine for character-scene interactions
Starke, S., Zhang, H., Komura, T., & Saito, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356505 · Cited by 291 · see citing works
Mode-adaptive neural networks for quadruped motion control
Zhang, H., Starke, S., Komura, T., & Saito, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201366 · Cited by 276 · see citing works
Learning motion manifolds with convolutional autoencoders
Holden, D., Saito, J., Komura, T., & Joyce, T. (2015) · SIGGRAPH Asia Technical Briefs · doi:10.1145/2820903.2820918 · Cited by 268 · see citing works
Learning Variable-Length Markov Models of Behavior
Galata, A., Johnson, N. J., & Hogg, D. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0894 · Cited by 267 · see citing works
Learning Video Representations using Contrastive Bidirectional TransformerPreprint
Sun, C., Baradel, F., Murphy, K. P., & Schmid, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05743 · Cited by 258 · see citing works
Cross View Fusion for 3D Human Pose Estimation
Qiu, H., Wang, C., Wang, J., Wang, N., & Zeng, W. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00444 · Cited by 241 · see citing works
Neural Kinematic Networks for Unsupervised Motion Retargetting
Villegas, R., Yang, J., Ceylan, D., & Lee, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00901 · Cited by 240 · see citing works
Learning Human Motion Models for Long-Term Predictions
Ghosh, P. S., Song, J., Aksan, E., & Hilliges, O. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00059 · Cited by 217 · see citing works
Heck, R., & Gleicher, M. L. (2007) · ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) · doi:10.1145/1230100.1230123 · Cited by 211 · see citing works
Multi-modal Dense Video Captioning
lashin, V., & Rahtu, E. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00487 · Cited by 200 · see citing works
Human Motion Prediction via Spatio-Temporal Inpainting
Hernandez, A., Gall, J., & Moreno, F. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00723 · Cited by 196 · see citing works
Structured Prediction Helps 3D Human Motion Modelling
Aksan, E., Kaufmann, M., & Hilliges, O. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00724 · Cited by 194 · see citing works
Benchmarking and Error Diagnosis in Multi-instance Pose Estimation
Ronchi, M. R., & Perona, P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.48 · Cited by 187 · see citing works
Local motion phases for learning multi-contact character movements
Starke, S., Zhao, Y., Komura, T., & Zaman, K. A. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392450 · Cited by 183 · see citing works
Action-Agnostic Human Pose Forecasting
Chiu, H.-K., Adeli, E., Wang, B., Huang, D.-A., & Niebles, J. C. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00156 · Cited by 163 · see citing works
Tang, T., Jia, J., & Mao, H. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240526 · Cited by 162 · see citing works
Shlizerman, E., Dery, L., Schoen, H., & Kemelmacher-Shlizerman, I. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00790 · Cited by 154 · see citing works
Leveraging Weakly Supervised Data to Improve End-to-end Speech-to-text Translation
Ye, J., Johnson, M., Macherey, W., Weiss, R. J., Cao, Y. J., Chiu, C., Ari, N., Laurenzo, S., & Wu, Y. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683343 · Cited by 153 · see citing works
Auto-Conditioned Recurrent Networks for Extended Complex Human Motion SynthesisPreprint
Li, Z., Zhou, Y., Xiao, S., He, C., Huang, Z., & Li, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05363 · Cited by 150 · see citing works
Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents
Bhattacharya, U., Rewkowski, N., Banerjee, A., Guhan, P., Bera, A., & Manocha, D. (2021) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr50410.2021.00037 · Cited by 144 · see citing works
DeepDance: Music-to-Dance Motion Choreography With Adversarial Learning
Sun, G., Wong, Y., Cheng, Z., Kankanhalli, M., Geng, W., & Li, X. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2981989 · Cited by 140 · see citing works
Investigating the use of recurrent motion modelling for speech gesture generation
Ferstl, Y., & McDonnell, R. (2018) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3267851.3267898 · Cited by 138 · see citing works
Weakly Supervised Dense Event Captioning in VideosPreprint
Duan, X., Huang, W., Gan, C., Wang, J., Zhu, W., & Huang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03849 · Cited by 131 · see citing works
Dancing‐to‐Music Character Animation
Shiratori, T., Nakazawa, A., & Ikeuchi, K. (2006) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2006.00964.x · Cited by 126 · see citing works
Foley Music: Learning to Generate Music from Videos
Gan, C., Huang, D. Y., Chen, P., Tenenbaum, J. B., & Torralba, A. B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_44 · Cited by 126 · see citing works
Cross-Modal Relation-Aware Networks for Audio-Visual Event Localization
Xu, H., Zeng, R., Wu, Q., Tan, M., & Gan, C. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413581 · Cited by 103 · see citing works
Imitation Learning for Human Pose Prediction
Wang, B., Adeli, E., Chiu, H.-K., Huang, D.-A., & Niebles, J. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00722 · Cited by 102 · see citing works
Ferreira, J., Coutinho, T., Gomes, T. L., NETO, J. F. L., Azevedo, R., Martins, R., & Nascimento, E. R. (2020) · Computers & Graphics · doi:10.1016/j.cag.2020.09.009 · Cited by 98 · see citing works
Dance Revolution: Long-Term Dance Generation with Music via Curriculum LearningPreprint
Huang, R., Huang, H., Wu, W., Sawada, K. B., Zhang, M., & Jiang, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06119 · Cited by 92 · see citing works
Example-Based Automatic Music-Driven Conventional Dance Motion Synthesis
Fan, R., Xu, S., & Geng, W. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.73 · Cited by 90 · see citing works
Efficient content-based retrieval of motion capture data
Müller, M., Röder, T., & Clausen, M. V. (2005) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1186822.1073247 · Cited by 87 · see citing works
Learning Statistical Models of Human Motion
Bowden, R. (2000) · Cited by 86 · see citing works
Tsuchida, S., Fukayama, S., Hamasaki, M., & Goto, M. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3527853 · Cited by 81 · see citing works
Du, X., Vasudevan, R., & Johnson‐Roberson, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2895266 · Cited by 80 · see citing works
Learning to Generate Diverse Dance Motions with TransformerPreprint
Li, J., Yin, Y., Chu, H., Zhou, Y., Wang, T., Fidler, S., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08171 · Cited by 75 · see citing works
Self-supervised Dance Video Synthesis Conditioned on Music
Ren, X., Li, H., Huang, Z., & Chen, Q. (2020) · ACM Multimedia · doi:10.1145/3394171.3413932 · Cited by 74 · see citing works
ChoreoNet: Towards Music to Dance Synthesis with Choreographic Action Unit
Ye, Z., Wu, H., Jia, J., Bu, Y., Chen, W., Meng, F., & Wang, Y. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414005 · Cited by 68 · see citing works
Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation
Yalta, N., Watanabe, S., Nakadai, K., & Ogata, T. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851872 · Cited by 57 · see citing works
Animating by multi-level sampling
Pullen, K., & Bregler, C. (2002) · Computer Animation (CA) · doi:10.1109/ca.2000.889031 · Cited by 51 · see citing works
Generative Autoregressive Networks for 3D Dancing Move Synthesis From Music
Ahn, H., Kim, J., Kim, K., & Oh, S. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2977333 · Cited by 51 · see citing works
Adversarial gesture generation with realistic gesture phasing
Ferstl, Y., Neff, M., & McDonnell, R. (2020) · Computers & Graphics · doi:10.1016/j.cag.2020.04.007 · Cited by 51 · see citing works
Temporally Guided Music-to-Body-Movement Generation
Kao, H.-K., & Su, L. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413848 · Cited by 40 · see citing works
FMDistance: A Fast and Effective Distance Function for Motion Capture Data
Onuma, K., Faloutsos, C., & Hodgins, J. K. (2008) · doi:10.2312/egs.20081027 · Cited by 37 · see citing works
Listen to Dance: Music-driven choreography generation using Autoregressive Encoder-Decoder NetworkPreprint
Lee, J., Kim, S., & Lee, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00818 · Cited by 35 · see citing works
Dance music, movement and tempo preferences
Moelants, D. (2003) · Ghent University Academic Bibliography (Ghent University) · Cited by 26 · see citing works
Kundu, J. N., Buckchash, H., Mandikal, P., Rahul, M., Jamkhandi, A., & Babu, R. V. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093627 · Cited by 20 · see citing works
Attention, please: A Spatio-temporal Transformer for 3D Human Motion PredictionPreprint
Aksan, E., Cao, P., Kaufmann, M., & Hilliges, O. (2020) · arXiv (Cornell University) · Cited by 18 · see citing works
Music2Dance: Music-driven Dance Generation using WaveNet.Preprint
Zhuang, W., Wang, C., Xia, S., Chai, J., & Wang, Y. (2020) · arXiv (Cornell University) · Cited by 17 · see citing works
Music-oriented Dance Video Synthesis with Pose Perceptual LossPreprint
Ren, X., Li, H., Huang, Z., & Chen, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.06606 · Cited by 16 · see citing works
Towards 3D Dance Motion Synthesis and ControlPreprint
Zhuang, W., Wang, Y., Robinson, J. P., Wang, C., Shao, M., Fu, Y., & Xia, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05743 · Cited by 15 · see citing works
Feel The Music: Automatically Generating A Dance For An Input SongPreprint
Tendulkar, P., Das, A., Kembhavi, A., & Parikh, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11905 · Cited by 5 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
4 of the 94 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE
Petrovich, M., Black, M. J., & Varol, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01080 · Cited by 506 · see citing works
MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355414 · Cited by 566 · see citing works
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437 · see citing works
TEMOS: Generating Diverse Human Motions from Textual Descriptions
Petrovich, M., Black, M. J., & Varol, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20047-2_28 · Cited by 366 · see citing works
Executing your Commands via Motion Diffusion in Latent Space
Chen, X., Jiang, B., Liu, W., Huang, Z., Fu, B., Chen, T., & Yu, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01726 · Cited by 370 · see citing works
MotionCLIP: Exposing Human Motion Generation to CLIP Space
Tevet, G., Gordon, B., Hertz, A., Bermano, A. H., & Cohen‐Or, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20047-2_21 · Cited by 331 · see citing works
Generating Human Motion from Textual Descriptions with Discrete Representations
Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., & Ying, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01415 · Cited by 328 · see citing works
EDGE: Editable Dance Generation From Music
Tseng, J., Castellon, R., & Liu, C. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00051 · Cited by 275 · see citing works
Li, Y., El Habib Daho, M., Conze, P.-H., Zeghlache, R., Le Boité, H., Tadayoni, R., Cochener, B., Lamard, M., & Quellec, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108635 · Cited by 274 · see citing works
PhysDiff: Physics-Guided Human Motion Diffusion Model
Yuan, Y., Song, J., Iqbal, U., Vahdat, A., & Kautz, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01467 · Cited by 246 · see citing works
Hong, F., Zhang, M., Pan, L., Cai, Z., Yang, L., & Liu, Z. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530094 · Cited by 218 · see citing works
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
Li, S., Yu, W., Gu, T., Lin, C., Wang, Q., Qian, C., Loy, C. C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01077 · Cited by 208 · see citing works
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Alexanderson, S., Nagy, R., Beskow, J., & Henter, G. E. (2023) · ACM Transactions on Graphics · doi:10.1145/3592458 · Cited by 206 · see citing works
3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting
Qian, Z., Wang, S., Mihajlovic, M., Geiger, A., & Tang, S. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.00480 · Cited by 279 · see citing works
MoFusion: A Framework for Denoising-Diffusion-Based Motion Synthesis
Dabral, R., Mughal, M. H., Golyanik, V., & Theobalt, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00941 · Cited by 174 · see citing works
Guided Motion Diffusion for Controllable Human Motion Synthesis
Karunratanakul, K., Preechakul, K., Suwajanakorn, S., & Tang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00205 · Cited by 172 · see citing works
BEAT: A Large-Scale Semantic and Emotional Multi-modal Dataset for Conversational Gestures Synthesis
Liu, H., Zhu, Z., Iwamoto, N., Peng, Y., Li, Z., Zhou, Y., Bozkurt, E., & Zheng, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_36 · Cited by 164 · see citing works
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
Ruan, L., Ma, Y., Yang, H., He, H., Liu, B., Fu, J., Yuan, N. J., Jin, Q., & Guo, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00985 · Cited by 154 · see citing works
DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer
Li, B., Zhao, Y., Shi, Z., & Sheng, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20014 · Cited by 143 · see citing works
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 162 · see citing works
BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion
Black, M. J., Patel, P., Tesch, J., & Yang, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00843 · Cited by 148 · see citing works
TEACH: Temporal Action Composition for 3D Humans
Athanasiou, N., Petrovich, M., Black, M. J., & Varol, G. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00053 · Cited by 141 · see citing works
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Yuan, Y., Iqbal, U., Molchanov, P. A., Kitani, K., & Kautz, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01076 · Cited by 133 · see citing works
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation
Zhu, L., Liu, X., Liu, X., Qian, R., Liu, Z., & Yu, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01016 · Cited by 131 · see citing works
State of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 156 · see citing works
Self-citationAo, T., Gao, Q., Lou, Y., Chen, B., & Liu, L. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555435 · Cited by 126 · see citing works
InterGen: Diffusion-Based Multi-human Motion Generation Under Complex Interactions
Liang, H., Zhang, W., Li, W., Jingyi Yu, A., & Xu, L. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02042-6 · Cited by 128 · see citing works
GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping
Taheri, O., Choutas, V., Black, M. J., & Tzionas, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01291 · Cited by 118 · see citing works
DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection
Tao, Y., Yang, M., Li, H., Wu, Y., & Hu, B. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3350071 · Cited by 114 · see citing works
Human Motion Generation: A Survey
Zhu, W., Ma, X., Ro, D., Ci, H., Zhang, J., Shi, J., Gao, F., Tian, Q. C., & Wang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3330935 · Cited by 118 · see citing works
SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos
Zeng, A., Yang, L., Ju, X., Li, J., Wang, J., & Xu, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_36 · Cited by 108 · see citing works
HumanMAC: Masked Motion Completion for Human Motion Prediction
Chen, L.-H., Zhang, J., Li, Y., Pang, Y., Xia, X., & Liu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00875 · Cited by 104 · see citing works
ARAH: Animatable Volume Rendering of Articulated Human SDFs
Wang, S., Schwarz, K., Geiger, A., & Tang, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_1 · Cited by 103 · see citing works
HuMMan: Multi-modal 4D Human Dataset for Versatile Sensing and Modeling
Cai, Z., Ren, D., Zeng, A., Lin, Z., Yu, T., Wang, W., Fan, X., Gao, Y., Yu, Y., Pan, L., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_33 · Cited by 96 · see citing works
Liu, H., Zhu, Z., Becherini, G., Peng, Y., Su, M., Zhou, Y., Zhe, X., Iwamoto, N., Zheng, B., & Black, M. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00115 · Cited by 107 · see citing works
COUCH: Towards Controllable Human-Chair Interactions
Zhang, X., Bhatnagar, B. L., Starke, S., Guzov, V., & Pons‐Moll, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_30 · Cited by 87 · see citing works
Disco: Disentangled Control for Realistic Human Dance Generation
Wang, T., Li, L., Lin, K., Zhai, Y., Lin, C.-C., Yang, Z., Zhang, H., Liu, Z., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00891 · Cited by 107 · see citing works
PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection
Chen, A., Zhang, K., Zhang, R., Wang, Z., Lu, Y., Guo, Y., & Zhang, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00512 · Cited by 85 · see citing works
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos
Liu, S., Tripathi, S., Majumdar, S., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00328 · Cited by 82 · see citing works
FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation
Li, R., Zhao, J., Zhang, Y., Su, M., Ren, Z., Zhang, H., Tang, Y., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00939 · Cited by 82 · see citing works
Self-citationMotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
Zhang, Y., Huang, D., Liu, B., Tang, S., Lu, Y., Chen, L., Bai, L., Chu, Q., Yu, N., & Ouyang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28567 · Cited by 89 · see citing works
Rhythm is a Dancer: Music-Driven Motion Synthesis With Global Structure
Aristidou, A., Yiannakidis, A., Aberman, K., Cohen‐Or, D., Shamir, A., & Chrysanthou, Y. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3163676 · Cited by 73 · see citing works
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Cheng, W., Chen, R., Fan, S., Yin, W., Chen, K., Cai, Z., Wang, J., Gao, Y., Yu, Z., Lin, Z., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01829 · Cited by 68 · see citing works
TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration
Gong, K., Lian, D., Chang, H., Guo, C., Jiang, Z., Zuo, X., Mi, M. B., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00912 · Cited by 68 · see citing works
Li, R., Zhang, Y.-X., Zhang, Y., Zhang, H., Guo, J., Zhang, Y. T., Liu, Y., & Li, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00151 · Cited by 64 · see citing works
Self-citationGIMO: Gaze-Informed Human Motion Prediction in Context
Zheng, Y., Yang, Y., Mo, K., Li, J., Yu, C., Liu, Y., Liu, C. K., & Guibas, L. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_39 · Cited by 63 · see citing works
Implicit Neural Representations for Variable Length Human Motion Generation
Cervantes, P., Sekikawa, Y., Sato, I., & Shinoda, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_22 · Cited by 62 · see citing works
A Brand New Dance Partner: Music-Conditioned Pluralistic Dancing Controlled by Multiple Dance Genres
Kim, J., Oh, H., Kim, S., Tong, H., & Lee, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00348 · Cited by 60 · see citing works
Multimodal fusion models for pulmonary embolism mortality prediction
Cahan, N., Klang, E., Marom, E. M., Soffer, S., Barash, Y., Burshtein, E., Konen, E., & Greenspan, H. K. (2023) · Scientific Reports · doi:10.1038/s41598-023-34303-8 · Cited by 62 · see citing works
MotionLCM: Real-Time Controllable Motion Generation via Latent Consistency Model
Dai, W., Chen, L.-H., Wang, J., Liu, J., Dai, B., & Tang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_22 · Cited by 61 · see citing works
UDE: A Unified Driving Engine for Human Motion Generation
Zhou, Z., & Wang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00545 · Cited by 57 · see citing works
Chen, J., Liu, Y., Wang, J., Zeng, A., Li, Y., & Chen, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00702 · Cited by 56 · see citing works
LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors
Ren, Y., Zhao, C., He, Y., Cong, P., Liang, H., Jingyi Yu, A., Xu, L., & Ma, Y. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3247088 · Cited by 52 · see citing works
Liu, H., Iwamoto, N., Zhu, Z., Li, Z., Zhou, Y., Bozkurt, E., & Zheng, B. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548400 · Cited by 51 · see citing works
SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation
Athanasiou, N., Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00916 · Cited by 50 · see citing works
Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation
Azadi, S., Shah, A., Hayes, T., Parikh, D., & Gupta, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01381 · Cited by 49 · see citing works
Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations
Noguchi, A., Sun, X. N., Lin, S., & Harada, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_36 · Cited by 48 · see citing works
Priority-Centric Human Motion Generation in Discrete Latent Space
Kong, H., Gong, K., Lian, D., Mi, M. B., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01360 · Cited by 48 · see citing works
Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model
Wang, Y., Leng, Z., Li, F. W. B., Wu, S., & Liang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02014 · Cited by 48 · see citing works
Music-Driven Group Choreography
Le, N., Pham, T., Do, T., Tjiputra, E., Tran, Q. D., & Nguyen, A. Q. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00838 · Cited by 45 · see citing works
HumanBench: Towards General Human-Centric Perception with Projector Assisted Pretraining
Tang, S., Chen, C., Xie, Q., Chen, M., Wang, Y., Ci, Y., Bai, L., Zhu, F., Yang, H., Li, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02104 · Cited by 45 · see citing works
Quantized GAN for Complex Music Generation from Dance Videos
Zhu, Y., Olszewski, K., Wu, Y., Achlioptas, P., Chai, M., Yan, Y., & Tulyakov, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_11 · Cited by 44 · see citing works
Vision + X: A Survey on Multimodal Learning in the Light of Data
Zhu, Y., Wu, Y., Sebe, N., & Yan, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3420239 · Cited by 46 · see citing works
Learning Locally Editable Virtual Humans
Ho, H.-I., Xue, L., Song, J., & Hilliges, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02014 · Cited by 43 · see citing works
EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation
Zhou, W., Dou, Z., Cao, Z., Liao, Z., Wang, J., Wang, W., Liu, Y., Komura, T., Wang, W., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72627-9_2 · Cited by 47 · see citing works
Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation
Petrovich, M., Litany, O., Iqbal, U., Black, M. J., Varol, G., Peng, X. B., & Rempe, D. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00197 · Cited by 44 · see citing works
The Wanderings of Odysseus in 3D Scenes
Zhang, Y., & Tang, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01983 · Cited by 41 · see citing works
CaV3: Cache-assisted Viewport Adaptive Volumetric Video Streaming
Liu, J., Zhu, B., Wang, F., Jin, Y., Zhang, W., Xu, Z., & Cui, S. R. (2023) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr55154.2023.00033 · Cited by 40 · see citing works
Gaussian Shell Maps for Efficient 3D Human Generation
Abdal, R., Wang, Y., Shi, Z., Xu, Y., Po, R., Kuang, Z., Chen, Q., Yeung, D., & Wetzstein, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00902 · Cited by 40 · see citing works
Zheng, J., Zheng, Q., Fang, L., Liu, Y., & Yi, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00064 · Cited by 38 · see citing works
Optimizing Diffusion Noise Can Serve As Universal Motion Priors
Karunratanakul, K., Preechakul, K., Aksan, E., Beeler, T., Suwajanakorn, S., & Tang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00133 · Cited by 42 · see citing works
Sárándi, I., Hermans, A., & Leibe, B. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00297 · Cited by 38 · see citing works
An Empirical Study on How People Perceive AI-generated Music
Chu, H., Kim, J., Kim, S., Lim, H., Lee, H., Jin, S., Lee, J., Kim, T.-H., & Ko, S. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557235 · Cited by 39 · see citing works
Interaction Transformer for Human Reaction Generation
Chopin, B., Tang, H., Otberdout, N., Daoudi, M., & Sebe, N. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3242152 · Cited by 37 · see citing works
Emotional Speech-Driven 3D Body Animation via Disentangled Latent Diffusion
Chhatre, K., Daněček, R., Athanasiou, N., Becherini, G., Peters, C., Black, M. J., & Bolkart, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00190 · Cited by 37 · see citing works
Video Background Music Generation: Dataset, Method and Evaluation
Zhuo, L., Wang, Z., Wang, B., Liao, Y., Bao, C., Peng, S., Han, S., Zhang, A., Fang, F., & Liu, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01433 · Cited by 36 · see citing works
PoseFix: Correcting 3D Human Poses with Natural Language
Delmas, G., Weinzaepfel, P., Moreno-Noguer, F., & Rogez, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01379 · Cited by 36 · see citing works
TLControl: Trajectory and Language Control for Human Motion Synthesis
Wan, W., Dou, Z., Komura, T., Wang, W., Jayaraman, D., & Liu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_3 · Cited by 45 · see citing works
Evaluating Gesture Generation in a Large-scale Open Challenge: The GENEA Challenge 2022
Kucherenko, T., Wolfert, P., Yoon, Y., Viegas, C., Nikolov, T., Tsakov, M., & Henter, G. E. (2024) · ACM Transactions on Graphics · doi:10.1145/3656374 · Cited by 37 · see citing works
Role-aware Interaction Generation from Textual Description
Tanaka, M., & Fujiwara, K. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01466 · Cited by 35 · see citing works
EgoHumans: An Egocentric 3D Multi-Human Benchmark
Khirodkar, R., Bansal, A., Ma, L., Newcombe, R., Vo, M., & Kitani, K. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01814 · Cited by 36 · see citing works
Bioinspired In-Sensor Multimodal Fusion for Enhanced Spatial and Spatiotemporal Association
Ma, S., Zhou, Y., Wan, T., Ren, Q., Yan, J., Fan, L., Yuan, H., Chan, M., & Chai, Y. (2024) · Nano Letters · doi:10.1021/acs.nanolett.4c01727 · Cited by 34 · see citing works
Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation
Wang, Y., Mu, L., Liu, J., Leng, Z., Li, F. W. B., Zhang, Z., & Liang, X. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02392-9 · Cited by 36 · see citing works
IDOL: Instant Photorealistic 3D Human Creation from a Single Image
Zhuang, Y., Lv, J., Wen, H., Shuai, Q., Zeng, A., Zhu, H., Chen, S., Yang, Y., Cao, X., & Liu, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02450 · Cited by 34 · see citing works
Seamless Human Motion Composition with Blended Positional Encodings
Barquero, G., Escalera, S., & Palmero, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00051 · Cited by 33 · see citing works
LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation
Zhi, Y., Cun, X., Chen, X., Shen, X., Guo, W. Q., Huang, S., & Gao, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01902 · Cited by 34 · see citing works
DanceConv: Dance Motion Generation with Convolutional Networks
Kritsis, K., Gkiokas, A., Pikrakis, A., & Katsouros, V. (2022) · IEEE Access · doi:10.1109/access.2022.3169782 · Cited by 35 · see citing works
Bailando ++: 3D Dance GPT With Choreographic Memory
Li, S., Yu, W., Gu, T., Lin, C., Wang, Q., Qian, C., Loy, C. C., & Liu, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3319435 · Cited by 32 · see citing works
Au, H.-Y., Chen, J., Jiang, J., & Guo, Y.-K. (2022) · ACM Multimedia · doi:10.1145/3503161.3547797 · Cited by 42 · see citing works
Perspectives on Advancing Multimodal Learning in Environmental Science and Engineering Studies.
Liu, W., Chen, J.-W., Wang, H.-B., Fu, Z.-Q., Peijnenburg, W., & Hong, H.-X. (2024) · Environmental Science and Technology · doi:10.1021/acs.est.4c03088 · Cited by 44 · see citing works
DanceGen: Supporting Choreography Ideation and Prototyping with Generative AI
Liu, Y., & Sra, M. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661594 · Cited by 30 · see citing works
A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision
Zhao, Z., Chai, W., Hao, S., Hu, W., Wang, G., Cao, S., Song, M., Hwang, J., & Wang, G. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3554801 · Cited by 30 · see citing works
With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition
Kazakos, E., Huh, J., Nagrani, A., Zisserman, A., & Damen, D. (2021) · British Machine Vision Conference (BMVC) · doi:10.5244/c.35.171 · Cited by 30 · see citing works
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 29 · see citing works
DeciWatch: A Simple Baseline for $$10\times $$ Efficient 2D and 3D Pose Estimation
Zeng, A., Ju, X., Yang, L., Gao, R., Zhu, X., Dai, B., & Xu, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_35 · Cited by 29 · see citing works
DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
Qi, Q., Zhuo, L., Zhang, A., Liao, Y., Fang, F., Liu, S., & Yan, S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612307 · Cited by 30 · see citing works
ActorsNeRF: Animatable Few-shot Human Rendering with Generalizable NeRFs
Mu, J., Sang, S., Vasconcelos, N. M., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01686 · Cited by 28 · see citing works
A Unified Framework for Real Time Motion Completion
Duan, Y., Lin, Y., Zou, Z., Yuan, Y. G., Qian, Z., & Zhang, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20368 · Cited by 28 · see citing works
Dance with You: The Diversity Controllable Dancer Generation via Diffusion Models
Yao, S., Sun, M., Li, B., Yang, F., Wang, J., & Zhang, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612046 · Cited by 29 · see citing works
- Authors
- Li, R., Yang, S., Ross, D. A., & Kanazawa, A.
- Affiliations
- University of Southern California (US)
- Google (United States) (US)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13381-13392
- OpenAlex
- W3204221554
- Cited by
- 553
- References
- 94
- Citations by year
- 2026: 92 · 2025: 146 · 2024: 154 · 2023: 113 · 2022: 46 · 2021: 2
- FWCI
- 117.49
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- cross-modal transformer, full attention, AIST, dance genres, choreography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
