Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation
Open accessAbstract
Audio-driven talking-head synthesis is a popular research topic for virtual human-related applications. However, the inflexibility and inefficiency of existing methods, which necessitate expensive end-to-end training to transfer emotions from guidance videos to talking-head predictions, are significant limitations. In this work, we propose the Emotional Adaptation for Audio-driven Talking-head (EAT) method, which transforms emotion-agnostic talking-head models into emotion-controllable ones in a cost-effective and efficient manner through parameter-efficient adaptations. Our approach utilizes a pretrained emotion-agnostic talking-head transformer and introduces three lightweight adaptations (the Deep Emotional Prompts, Emotional Deformation Network, and Emotional Adaptation Module) from different perspectives to enable precise and realistic emotion controls. Our experiments demonstrate that our approach achieves state-of-the-art performance on widely-used benchmarks, including LRW and MEAD. Additionally, our parameter-efficient adaptations exhibit remarkable generalization ability, even in scenarios where emotional training videos are scarce or nonexistent. Project website: https://yuangan.github.io/eat/
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pioneer talking-head generation works typically utilize biometric signals such as facial keypoints or 3D priors as intermediate motion representation to animate the reference face while ensuring lip synchronization.
“pioneer talking-head generation works typically utilize biometric signals such as facial keypoints [22, 28, 30, 45], or 3D priors [11, 13, 16, 23, 32, 43, 44] as intermediate motion representation to animate the reference face while ensuring lip synchronization.”
From MoCha: Towards Movie-Grade Talking Character Synthesis (2025) · cites this paper as [13]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019 · see citing works
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 1× · see citing works
“Recently, prompt [29] has attracted more attention in vision task.”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 1× · see citing works
“Recently, prompt [29] has attracted more attention in vision task.”
BackgroundLearning to Prompt for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01653-1 · Cited by 3,304 · cited 2× · see citing works
“Inspired by prompt tuning in the language model, prompt tuning recently has been proposed in various visual tasks [38, 20, 56, 21, 55, 13, 27] for effectiveness and efficiency.”
BackgroundViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767 · see citing works
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338 · see citing works
Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00320 · Cited by 2,294 · see citing works
VoxCeleb2: Deep Speaker Recognition
Chung, J. S., Nagrani, A., & Zisserman, A. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1929 · Cited by 2,256 · see citing works
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977 · see citing works
Self-citationJia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965 · see citing works
Conditional Prompt Learning for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01631 · Cited by 1,918 · see citing works
StarGAN v2: Diverse Image Synthesis for Multiple Domains
Choi, Y., Uh, Y., Yoo, J., & Ha, J.-W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00821 · Cited by 1,663 · see citing works
Multiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407 · see citing works
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery
Patashnik, O., Wu, Z., Shechtman, E., Cohen‐Or, D., & Lischinski, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00209 · Cited by 1,079 · see citing works
CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset
Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2336244 · Cited by 954 · see citing works
Mel Frequency Cepstral Coefficients For Music Modeling.
Logan, B. (2000) · doi:10.5281/zenodo.1416443 · Cited by 928 · see citing works
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897 · see citing works
Out of Time: Automated Lip Sync in the Wild
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54427-4_19 · Cited by 689 · see citing works
GANimation: Anatomically-Aware Facial Animation from a Single Image
Pumarola, A., Agudo, A., Martinez, A. M., Sanfeliu, A., & Moreno-Noguer, F. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_50 · Cited by 621 · see citing works
Learning multiple visual domains with residual adapters
Rebuffi, S.-A., Bilen, H., & Vedaldi, A. (2017) · arXiv · doi:10.48550/arxiv.1705.08045 · Cited by 605 · see citing works
First Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585 · see citing works
Emotional Expression: Advances in Basic Emotion Theory
Keltner, D. J., Sauter, D. A., Tracy, J. L., & Cowen, A. (2019) · Journal of Nonverbal Behavior · doi:10.1007/s10919-019-00293-3 · Cited by 537 · see citing works
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54184-6_6 · Cited by 537 · see citing works
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
Wang, T.-C., Mallya, A., & Liu, M.-Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00991 · Cited by 468 · see citing works
Hierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss
Chen, L., Maddox, R. K., Duan, Z., & Xu, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00802 · Cited by 437 · see citing works
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
Zhou, H., Liu, Y., Liu, Z., Luo, P., & Wang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019299 · Cited by 416 · see citing works
Zhou, Y., Han, X., Shechtman, E., Echevarria, J., Kalogerakis, E., & Li, D. (2020) · ACM Transactions on Graphics · doi:10.1145/3414685.3417774 · Cited by 403 · see citing works
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
Guo, Y., Chen, K., Liang, S., Liu, Y.-J., Bao, H., & Zhang, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00573 · Cited by 384 · see citing works
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
Zhou, H., Sun, Y., Wu, W., Loy, C. C., Wang, X., & Liu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00416 · Cited by 377 · see citing works
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model
Du, Y., Wei, F., Zhang, Z., Shi, M., Gao, Y., & Li, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01369 · Cited by 368 · see citing works
MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation
Wang, K., Wu, Q., Song, L., Yang, Z., Wu, W., Qian, C., He, R., Qiao, Y., & Loy, C. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_42 · Cited by 344 · see citing works
Self-citationYang, Y. P., Zhuang, Y., & Pan, Y. (2021) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2100463 · Cited by 310 · see citing works
Self-citationAudio-Driven Emotional Video Portraits
Ji, X., Zhou, H., Wang, K., Wu, W., Loy, C. C., Cao, X., & Xu, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01386 · Cited by 223 · see citing works
Frame Attention Networks for Facial Expression Recognition in Videos
Meng, D., Peng, X., Wang, K., & Qiao, Y. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803603 · Cited by 216 · see citing works
Talking Face Generation by Conditional Recurrent Adversarial Network
Song, Y., Zhu, J., Li, D., Wang, A. Y.-D., & Qi, H. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/129 · Cited by 192 · see citing works
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
Zhang, J. O., (Sasha), A. S., Zamir, A., Guibas, L. J., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_41 · Cited by 190 · see citing works
Associating Objects with Transformers for Video Object SegmentationPreprint
Yang, Z., Wei, Y., & Yang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02638 · Cited by 183 · see citing works
Self-citationTinyTL: Reduce Memory, Not Parameters for Efficient On-Device LearningPreprint
Cai, H., Gan, C., Zhu, L., & Han, S. (2020) · arXiv (Cornell University) · Cited by 160 · see citing works
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model
Ji, X., Zhou, H., Wang, K., Wu, Q., Wu, W., Xu, F., & Cao, X. (2022) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings · doi:10.1145/3528233.3530745 · Cited by 155 · see citing works
Expressive Talking Head Generation with Granular Audio-Visual Control
Liang, B., Pan, Y. M., Guo, Z., Zhou, H., Hong, Z., Han, X., Han, J., Liu, J., Ding, E., & Wang, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00338 · Cited by 124 · see citing works
Segment and Track AnythingPreprint
Cheng, Y., Li, L., Xu, Y., Li, X., Yang, Z., Wang, W., & Yang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06558 · Cited by 122 · see citing works
Self-citationDecoupling Features in Hierarchical Propagation for Video Object Segmentation
Yang, Z., & Yang, Y. (2022) · neural information processing systems · doi:10.52202/068431-2632 · Cited by 121 · see citing works
Self-citationOne-Shot Talking Face Generation from Single-Speaker Audio-Visual Correlation Learning
Wang, S., Li, L., Ding, Y., & Yu, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20154 · Cited by 115 · see citing works
Kum, S., & Nam, J. (2019) · Applied Sciences · doi:10.3390/app9071324 · Cited by 81 · see citing works
Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos
Li, M., Chen, L., Duarr, Y., Hu, Z., Feng, J., Zhou, J., & Lu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01926 · Cited by 80 · see citing works
CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentation
Li, K., Yang, Z., Chen, L., Yang, Y., & Xiao, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611724 · Cited by 69 · see citing works
Self-citationEmotion-Controllable Generalized Talking Face Generation
Sinha, S., Biswas, S., Yadav, R., & Bhowmick, B. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/184 · Cited by 50 · see citing works
Song, L., Wu, W., Fu, C., Qian, C., Loy, C. C., & He, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00227 · Cited by 26 · see citing works
In-N-Out Generative Learning for Dense Unsupervised Video Segmentation
Pan, X., Li, P. P., Yang, Z., Zhou, H., Zhou, C., Yang, H., Zhou, J., & Yang, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547909 · Cited by 8 · see citing works
Self-citationInstance as Identity: A Generic Online Paradigm for Video Instance Segmentation
Zhu, F., Yang, Z., Yu, X., Yang, Y., & Wei, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19818-2_30 · Cited by 8 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
2 of the 59 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis
Tan, S., Ji, B., Bi, M., & Pan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_23 · Cited by 45 · see citing works
Deepfake Generation and Detection: A Benchmark and Survey
Pei, G., Zhang, J., Hu, M., Zhang, Z., Wang, C., Wu, Y., Zhai, G., Yang, J. J., & Tao, D. (2026) · ACM Computing Surveys · doi:10.1145/3801962 · Cited by 113 · see citing works
Tan, S., Ji, B., & Pan, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02486 · Cited by 26 · see citing works
A Review of Human Emotion Synthesis Based on Generative Technology
Ma, F., Xie, Y., Li, Y., He, Y., Zhang, Y., Ren, H., Liu, Z., Yao, W. J., Ren, F., Yu, F. R., et al. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3573878 · Cited by 33 · see citing works
CapHuman: Capture Your Moments in Parallel Universes
Liang, C., Ma, F., Zhu, L., Deng, Y., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00612 · Cited by 20 · see citing works
Self-citationAniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding
Liu, T., Chen, F., Fan, S., Du, C., Chen, Q., Chen, X., & Yu, K. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681198 · Cited by 21 · see citing works
TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Styles
Ma, Y., Wang, S., Ding, Y., Ma, B., Lv, T., Fan, C., Hu, Z., Deng, Z., & Yu, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3581808 · Cited by 29 · see citing works
Exploring Interpretability in Deep Learning for Affective Computing: A Comprehensive Review
Zhang, X., Zhang, T., Sun, L., Zhao, J., & Jin, Q. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3723005 · Cited by 15 · see citing works
Self-citationOne-shot motion talking head generation with audio-driven model
Tang, P., Zhao, H., Meng, W., & Wang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129344 · Cited by 12 · see citing works
AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation
Sun, Y., Chu, W., Zhou, H., Wang, K., & Koike, H. (2024) · IEEE Access · doi:10.1109/access.2024.3390182 · Cited by 13 · see citing works
EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head
He, Q., Ji, X., Gong, Y., Lu, Y., Diao, Z., Huang, L., Yao, Y., Zhu, S., Ma, Z., Xu, S., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72998-0_4 · Cited by 11 · see citing works
InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generation
Wang, Y., Guo, J., Bai, J., Yu, R., He, T., Tan, X., Sun, X., & Bian, J. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i8.32877 · Cited by 14 · see citing works
InsTaG: Learning Personalized 3D Talking Head from Few-Second Video
Li, J., Zhang, J., Xiao, B., Zheng, J., Zhou, J. W., & Gu, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00999 · Cited by 9 · see citing works
SingingHead: A Large-Scale 4D Dataset for Singing Head Animation
Wu, S., Li, Y., Zhang, W., Jia, J., Zhu, Y., Yan, Y., Zhai, G., & Yang, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3623560 · Cited by 10 · see citing works
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment
Bounareli, S., Tzelepis, C., Argyriou, V., Patras, I., & Tzimiropoulos, G. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099159 · Cited by 14 · see citing works
EmoSpeaker: One-Shot Fine-Grained Emotion-Controlled Talking Face Generation
Feng, G., Cheng, H., Li, Y., Ma, Z., Li, C., Qian, Z., Miao, Q., & Pun, C. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3651074 · Cited by 6 · see citing works
Xu, Z., Chen, T., Yang, Z., Qing, C., Shi, Y., & Lin, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681017 · Cited by 5 · see citing works
Self-citationTalking Head Generation via Viewpoint and Lighting Simulation Based on Global Representation
Biao, D., & Zhang, L. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755503 · Cited by 4 · see citing works
Tu, G., Wang, B., Cambria, E., Li, W., & Xu, R. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3715200 · Cited by 4 · see citing works
Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation
Tan, W., Lin, C., Xu, C., Xu, F., Hu, X., Ji, X., Zhu, J., Wang, C., & Fu, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755286 · Cited by 4 · see citing works
Zhang, C., Wang, C., Zhang, J., Xu, H., Song, G., Xie, Y., Luo, L., Tian, Y., Feng, J., & Guo, X. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450491 · Cited by 4 · see citing works
Wu, Y., Deng, X., Shao, H., Cheng, K., Zhang, M., Jiang, Y., & Wang, F. (2025) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2025.066647 · Cited by 5 · see citing works
ListenFormer: Responsive Listening Head Generation with Non-autoregressive Transformers
Liu, M., Wang, J., Qian, X., & Li, H. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681182 · Cited by 3 · see citing works
ETAU: Towards Emotional Talking Head Generation Via Facial Action Unit
Lyu, J., Lan, X., Hu, G., Jiang, H., Gan, W., & Xue, J. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687525 · Cited by 3 · see citing works
LES-Talker: Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Space
Feng, G., Qian, Z., Li, Y., Jin, S., Miao, Q., & Pun, C. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3676758 · Cited by 3 · see citing works
An approach to optimizing semantic consistency for text-to-digital human generation
Liu, X., Liu, X., Yang, P., Wang, Z., & Liu, F. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111909 · Cited by 2 · see citing works
TH-Mamba: Spatial-Temporal Correlation Learning for Mamba-Based Talking Head Generation
Xu, X., Yu, Z., Jiang, K., Feng, X., & Lin, C. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3596747 · Cited by 2 · see citing works
Kim, S., & Song, B. C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73039-9_20 · Cited by 2 · see citing works
Multimodal Emotional Talking Face Generation Based on Action Units
Lyu, J., Lan, X., Hu, G., Jiang, H., Gan, W., Wang, J., & Xue, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3523359 · Cited by 2 · see citing works
Enhanced Temporal Representation and Spatial Alignment for High-Fidelity Talking Video Generation
Dong, B., & Zhang, L. (2025) · The Visual Computer · doi:10.1007/s00371-025-03999-2 · Cited by 2 · see citing works
EmoGene: Audio-Driven Emotional 3D Talking-Head Generation
Wang, W., & Fu, Y. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099460 · Cited by 2 · see citing works
Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation
Hu, G., Wei, J., Song, S., Kollias, D., Yang, X., Sun, Z., & Kaloidas, O. (2024) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb62174.2024.10744499 · Cited by 1 · see citing works
Dai, Q., Feng, H., Cui, W., Cai, X., Zheng, Y., & Zeng, M. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733322 · Cited by 1 · see citing works
Prompt-to-Animation: Generating Cognitively-Grounded Facial Expressions with LLMs
Durupınar, F., & Normoyle, A. (2025) · ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) · doi:10.1145/3769047.3769055 · Cited by 1 · see citing works
Toward Unified Co-Speech Gesture Generation via Hierarchical Implicit Periodicity Learning
Guo, X., Zhao, Y., & Li, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3645572 · Cited by 1 · see citing works
Emotionally Controllable Audio-driven Talking Face Generation
Xu, Y., Zhao, S., Liu, S., Xu, T., & Chen, E. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3779219 · Cited by 1 · see citing works
DH-FaceVid-1K: A Large-Scale High-Quality Dataset for Face Video Generation
Di, D., He, F., Sun, W., Ma, Y., Li, H., Chen, W., Fan, L., Su, T., & Yang, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01127 · Cited by 1 · see citing works
Emotional Synchronization for Audio-Driven Talking-Head Generation
Zhang, Z., Luo, Y., Zuo, Z., Hong, R., Yang, Y., & Wang, M. (2024) · Proceedings · doi:10.1109/icdm59182.2024.00068 · Cited by 1 · see citing works
Self-citationMoCha: Towards Movie-Grade Talking Character Synthesis
Wei, C., Sun, B., Ma, H., Hou, J., Juefei-Xu, F., He, Z., Dai, X., Zhang, L., Li, K., Hou, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.23307
“…audio sequence and a reference face, pioneer talking-head generation works typically utilize biometric signals such as facial keypoints [22, 28, 30, 45], or 3D priors [11, 13, 16, 23, 32, 43, 44] as intermediate motion representation to animate the reference face while ensuring lip synchronization.”
Emotion-Aware Talking Face Generation Based on 3DMM
Chen, X., & Tang, S. (2024) · International Conference on Neural Networks, Information and Communication Engineering (NNICE) · doi:10.1109/nnice61279.2024.10498924
Efficient Emotional Talking Head Generation via Dynamic 3D Gaussian Rendering
Liu, T., Li, J., Xiao, B., & Zheng, J. J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8508-7_6
DFNeRF: Disentangled Facial Neural Radiance Fields for Text-based Editing of Free-view Talking Head
Chen, B., Li, X., Wang, X., Zhang, Q., & Wang, H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889463
Sinha, S., Bhowmick, B., Tiwari, L., & Chanda, S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00474
Xiao, Z., Liu, X., Guo, J., He, J., Hong, R., & Wang, M. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3742790
Visual Content Generation in the Era of Large Foundation Models
Qu, L., Shen, F., Zhou, Z., Lyu, J., Wang, W., & Jiang, L. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3734580
EmoGaussian High-Fidelity Emotional Talking Head Generation with 3D Gaussian Splatting
Zhang, T., Xiao, Z., Guo, J., & Liu, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9856-1_10
Audio-Driven Talking Head Generation with Emotion Based on FLAME Geometry Model
Li, Y., Wang, L., Yu, Y., & Shen, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04555-3_15
Zhao, S., Wang, C., Zhao, W., Li, Z., Li, M., & Zhang, S. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755093
The ACM Multimedia 2025 Grand Challenge of Avatar-based Multimodal Empathetic Conversation
Zhang, H., Fei, H., Han, H. W., Liao, L., Cambria, E., & Zhang, M. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3762028
STSA: Spatial-Temporal Semantic Alignment for Visual Dubbing
Ding, Z., Xiong, M., Zhu, C., & Chen, J. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209490
Audio-Driven Emotion-Aware 3D Talking Face Generation from Single Image
Qiu, C.-S., Liu, F., Fu, H., Zhang, F., Cao, Y., Lai, Y., & Gao, L. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209335
EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters
Shen, X., Cai, H., Yu, D., Shen, W., Xu, Q., & Xue, X. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209861
DP-GaussTalk: Dual-Path Audio-Driven Feature Fusion for 3D Gaussian-Based Talking Head Synthesis
Chen, Z., Wang, L., & Yu, Y. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343340
VTalker: Text-Driven Synthesis of Talking Head with Vision Diffusion Transformer
Cai, Y., Qiao, P., & Li, D. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3793546
PoseTalk: Exploring Text- and Audio-Based Pose Control for One-Shot Talking Face Generation
Ling, J., Wang, Y., Xue, H., Xie, R., & Song, L. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3788691 · Cited by 2 · see citing works
RealTalk: Realistic Emotion-Aware Lifelike Talking-Head Synthesis
Wang, W., & Fu, Y. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00159
Transparent System For Lightweight Neural Talking Faces
Bartosz, O., Dąbała, Ł., & Będkowski, J. (2025) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip67698.2025.11396896
Fine-grained talking motion consistent network for realistic face generation
Xiao, Z., Guo, J., Liu, X., & He, J. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114364
An Analysis of the Human Ability to Detect Deepfakes With Geopolitical Content
Brienza, M., Golotta, M., Romano, M., Nardi, D., & Bloisi, D. D. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2026.3667623
Learning Spatial-Temporal Coherent Correlations for Speech-Preserving Facial Expression Manipulation
Chen, T., Lin, J., Yang, Z., Qing, C., Wang, G., & Lin, L. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3687518
Self-citationAashish Chandra, K., Aashutosh, A. V., & Das, A. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00134
Zhu, Y., Zhao, Z., Lin, J., Yang, Z., Qin, J., & Chen, T. (2026) · International Conference on Electronics Technology and Artificial Intelligence (ETAI) · doi:10.1109/etai68332.2026.11485380
Self-citationDiTalker: A unified DiT-based framework for high-quality and style-controllable portrait animation
He, F., F, Y., Fan, L., Di, D., & Su, T. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104819
Talking-Head Generation in Practice
Zhang, Z., Wang, L., Gao, Y., & Zhang, Y. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3794684 · Cited by 1 · see citing works
Jiang, Y., Cun, X., Zhang, Y., He, T., Tang, F., & Pun, C. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3699022
EmoPoseFace: Head Pose Aware Speech-Driven 3D Emotional Facial Animation Using Latent Diffusion
Zhao, X., Dai, J. P., Zhou, F., Wang, H., Song, Z., Hao, A., Qin, H., & Gao, Y. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3702133
Decoupling Vocal and Rhythmic Conditioning for Music-Driven Singing Avatar Animation
Jiang, Y., Cun, X., Feng, C.-B., Sun, J., & Pun, C. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810854
EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence
Li, Y., Yu, Y., Wang, L., & Shen, S. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810860
Controllable and temporally stable audio-driven talking-head generation for multi-user interactions
Yang, M., & Yang, H. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115528
Towards Unified Affective AI: A Cross-Modal Survey of Emotion Recognition, Generation, and Control
Mobbs, R., Makris, D., & Argyriou, V. (2026) · Applied Sciences · doi:10.3390/app16189333
Salahudeen, R., Fonkam, M. M., Vajjhala, N. R., Junaidu, S. B., & Garba, A. (2026) · Systems · doi:10.3390/systems14101198
Hu, C., Huang, Q., Shu, F., Li, D., & Huang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37966-5_1
Liu, J., Liu, K., Li, J., Yi, X., & Cao, Y. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11730078
- Authors
- Gan, Y., Yang, Z., Yue, X., Sun, L., & Yang, Y.
- Affiliations
- Zhejiang University (CN)
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 22577-22588
- OpenAlex
- W4390872769
- Cited by
- 72
- References
- 59
- Citations by year
- 2026: 22 · 2025: 35 · 2024: 15
- FWCI
- 7.70
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Fundamental Research Funds for the Central Universities
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- emotion control, parameter-efficient fine-tuning, virtual humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
