The DOI matches its Crossref record; 57 of the 58 references it cites were found in a registry or an index.
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation
Open accessAbstract
Talking head synthesis is a promising approach for the video production industry. Recently, a lot of effort has been devoted in this research area to improve the generation quality or enhance the model generalization. However, there are few works able to address both issues simultaneously, which is essential for practical applications. To this end, in this paper, we turn attention to the emerging powerful Latent Diffusion Models, and model the Talking head generation as an audio-driven temporally coherent denoising process (DiffTalk). More specifically, instead of employing audio signals as the single driving factor, we investigate the control mechanism of the talking face, and incorporate reference face images and landmarks as conditions for personality-aware generalized synthesis. In this way, the proposed DiffTalk is capable of producing high-quality talking head videos in synchronization with the source audio, and more importantly, it can be naturally generalized across different identities without further finetuning. Additionally, our DiffTalk can be gracefully tailored for higher-resolution synthesis with negligible extra computational cost. Extensive experiments show that the proposed DiffTalk efficiently synthesizes high-fidelity audio-driven talking head videos for generalized novel identities. For more video results, please refer to https://sstzal.github.io/DiffTalk/.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We investigated several face reenactment strategies including EAMM [26], AVFR-GAN [2], DiffTalk [54], ADNeRF [20] and ATVGnet [10] and concluded that these methods are not well suited for zero-shot lip-synced generation of unseen speakers.”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [54]
In the citing paper
“pioneer talking-head generation works typically utilize biometric signals such as facial keypoints [22, 28, 30, 45], or 3D priors [11, 13, 16, 23, 32, 43, 44] as intermediate motion representation to animate the reference face while ensuring lip synchronization.”
From MoCha: Towards Movie-Grade Talking Character Synthesis (2025) · cites this paper as [30]
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514
Denoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507
Cited by
Sorted by citations
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 197
State of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 198
Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis
Li, J., Zhang, J., Bai, X. P., Zhou, J. W., & Gu, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00696 · Cited by 131
Self-citationFace Generation and Editing With StyleGAN: A Survey
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3350004 · Cited by 105
Tian, L., Wang, Q. J., Zhang, B., & Bo, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73010-8_15 · Cited by 211
Tian, L., Wang, Q., Zhang, B., & Bo, L. (2024) · arXiv · doi:10.48550/arxiv.2402.17485 · Cited by 92
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Cai, Z., Ghosh, S., Adatia, A. P., Hayat, M., Dhall, A., Gedeon, T., & Stefanov, K. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3680795 · Cited by 59
“The variation in the number of subjects and videos in different sets is presented in Table 4 and Figure 4.”
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models
Aneja, S., Thies, J., Dai, A., & NieBner, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02009 · Cited by 62
EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis
Tan, S., Ji, B., Bi, M., & Pan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_23 · Cited by 45
SD-NeRF: Towards Lifelike Talking Head Animation via Spatially-Adaptive Dual-Driven NeRFs
Shen, S., Li, W., Huang, X., Zhu, Z. D., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3308441 · Cited by 39
Self-citationSpeech driven video editing via an audio-conditioned diffusion model
Bigioi, D., Basak, S., Stypułkowski, M., Zięba, M., Jordan, H., McDonnell, R., & Corcoran, P. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.104911 · Cited by 49
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization
Mukhopadhyay, S., Suri, S., Gadde, R. T., & Shrivastava, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00521 · Cited by 49
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cui, J., Li, H., Zhan, Y., Shang, H. L., Cheng, K., Ma, Y., Mu, S., Zhou, H., Wang, J., & Zhu, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01964 · Cited by 85
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cui, J., Li, H., Zhan, Y., Shang, H., Cheng, K., Ma, Y., Mu, S., Zhou, H., Wang, J., & Zhu, S. (2024) · arXiv · doi:10.48550/arxiv.2412.00733 · Cited by 32
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 32
PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models
Zhang, Y., Xing, Z., Zeng, Y., Fang, Y., & Chen, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00740 · Cited by 42
Li, J., Zhang, J., Bai, X. P., Zheng, J., Zhou, J. W., & Gu, L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102456 · Cited by 22
Self-citationHuman Motion Video Generation: A Survey
Xue, H., Luo, X., Hu, Z., Zhang, X., Xiang, X., Dai, Y., Liu, J., Zhang, Z., Li, M., Yang, J. J., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3594034 · Cited by 29
D 4 M: Dataset Distillation via Disentangled Diffusion Model
Su, D., Hou, J., Gao, W., Tian, Y., & Tang, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00555 · Cited by 33
DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generation
Hong, F.-T., Shen, L., & Xu, D. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339964 · Cited by 27
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions
Liu, Y., Lin, L., Yu, F., Zhou, C., & Li, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02104 · Cited by 20
AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding
Liu, T., Chen, F., Fan, S., Du, C., Chen, Q., Chen, X., & Yu, K. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681198 · Cited by 21
DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction
Xiong, J., Zhang, P., You, T., Li, C., Huang, W., & Zha, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02575 · Cited by 21
Application of a 3D Talking Head as Part of Telecommunication AR, VR, MR System: Systematic Review
Christoff, N., Neshov, N. N., Tonchev, K., & Manolova, A. H. (2023) · Electronics · doi:10.3390/electronics12234788 · Cited by 21
Audio-Driven Facial Animation with Deep Learning: A Survey
Jiang, D., Chang, J. M., You, L., Bian, S., Kosk, R., & Maguire, G. (2024) · Information · doi:10.3390/info15110675 · Cited by 23
LaMD: Latent Motion Diffusion for Image-Conditional Video Generation
Hu, Y., Chen, Z., & Luo, C. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02386-7 · Cited by 43
Wav2Lip‐HR: Synthesising clear high‐resolution talking head in the wild
Liang, C., Wang, Q., Chen, Y., & Tang, M. (2023) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2226 · Cited by 16
Sonic: Shifting Focus to Global Audio Perception in Portrait Animation
Ji, X., Hu, X., Xu, Z., Zhu, J., Lin, C., He, Q., Zhang, J., Luo, D., Chen, Y., Lin, Q., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00027 · Cited by 21
Revisiting Generalizability in Deepfake Detection: Improving Metrics and Stabilizing Transfer
Kamat, S., Agarwal, S., Darrell, T., & Rohrbach, A. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00049 · Cited by 16
EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model
Zhang, B., Zhang, X., Cheng, N., Yu, J., Xiao, J. Z., & Wang, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447505 · Cited by 13
AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis
Li, D., Zhao, K., Wang, W., Peng, B., Zhang, Y., Dong, J., & Tan, T. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28086 · Cited by 13
Increasing Importance of Joint Analysis of Audio and Video in Computer Vision: A Survey
Shahabaz, A., & Sarkar, S. (2024) · IEEE Access · doi:10.1109/access.2024.3391817 · Cited by 13
Seeing the Sound: Multilingual Lip Sync for Real-Time Face-to-Face Translation
Oskooei, A. R., Aktaş, M. S., & Keleş, M. (2024) · Computers · doi:10.3390/computers14010007 · Cited by 13
Audio-Semantic Enhanced Pose-Driven Talking Head Generation
Liu, M., Li, D., Li, Y., Song, X., & Nie, L. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3414412 · Cited by 11
3D facial modeling, animation, and rendering for digital humans: A survey
Zhang, Y., Su, R., Yu, J., & Li, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128168 · Cited by 13
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing
Xiong, L., Cheng, X., Tan, J., Wu, X., Li, X., Zhu, L., Ma, F., Li, M., Xu, H., & Hu, Z. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681108 · Cited by 10
SingingHead: A Large-Scale 4D Dataset for Singing Head Animation
Wu, S., Li, Y., Zhang, W., Jia, J., Zhu, Y., Yan, Y., Zhai, G., & Yang, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3623560 · Cited by 10
PersonaTalk: Bring Attention to Your Persona in Visual Dubbing
Zhang, L., Liang, S., Ge, Z., & Hu, T. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687618 · Cited by 9
Hierarchical Feature Warping and Blending for Talking Head Animation
Zhang, J., Liu, C., Xian, K., & Cao, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3375330 · Cited by 9
DiffDub: Person-Generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-Encoder
Liu, T., Du, C., Fan, S., Chen, F., & Yu, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446049 · Cited by 9
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment
Bounareli, S., Tzelepis, C., Argyriou, V., Patras, I., & Tzimiropoulos, G. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099159 · Cited by 17
Multimodal Fusion for Talking Face Generation Utilizing Speech-Related Facial Action Units
Liu, Z., Liu, X., Chen, S., Liu, J., Wang, L., & Bi, C. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3672565 · Cited by 7
Audio-Driven Talking Face Generation with Stabilized Synchronization Loss
Yaman, D., Eyiokur, F. İ., Bärmann, L., Ekenel, H. K., & Waibel, A. H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72655-2_24 · Cited by 7
Multimodal Feature-Guided Audio-Driven Emotional Talking Face Generation
Wang, X., Huo, Y., Liu, Y., Guo, X., Yan, F., & Zhao, G. (2025) · Electronics · doi:10.3390/electronics14132684 · Cited by 8
Decoupled Two-Stage Talking Head Generation via Gaussian-Landmark-Based Neural Radiance Fields
Ma, B., Cao, Y., & Zhang, L. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450482 · Cited by 7
ReSyncer: Rewiring Style-Based Generator for Unified Audio-Visually Synced Facial Performer
Guan, J., Xu, Z., Zhou, H., Wang, K., He, S., Zhang, Z., Liang, B., Feng, H., Ding, E., Liu, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72940-9_20 · Cited by 7
Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generation
Ye, Z., Chen, Z., Li, T., Huang, Z., Luo, W., & Qi, G.-J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02180 · Cited by 6
VIIS: Visible and Infrared Information Synthesis for Severe Low-Light Image Enhancement
Zhao, C., Yu, M., Yang, F., & Jing, P. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00218 · Cited by 7
DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis
Deng, Y., Zheng, D., Xie, J., Wang, J., Xie, W., Shen, L., & Song, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890278 · Cited by 5
FD2Talk: Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model
Yao, Z., Cheng, X., & Huang, Z. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681238 · Cited by 5
Rakesh, V. K., Mazumdar, S., Pal, S., Das, A. K., & Samanta, T. (2025) · The Visual Computer · doi:10.1007/s00371-025-04232-w · Cited by 12
Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation
Tan, J., Cheng, X., Xiong, L., Zhu, L., Li, X., Wu, X., Gong, K., Li, M., & Cai, Y. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688218 · Cited by 4
Zhang, C., Wang, C., Zhang, J., Xu, H., Song, G., Xie, Y., Luo, L., Tian, Y., Feng, J., & Guo, X. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450491 · Cited by 4
Learning and Evaluating Human Preferences for Conversational Head Generation
Zhou, M., Bai, Y., Zhang, W., Yao, T., Zhao, T., & Mei, T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612831 · Cited by 4
Gao, Y., Li, J., Chu, L., & Lü, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01824 · Cited by 4
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance
Zhang, H., Yuan, Z., Zheng, C., Yan, X., Wang, B., Li, G., Wu, S., Cui, S. R., & Li, Z. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3566382 · Cited by 3
GaPTalk: precision-controlled 3D Gaussian rendering for personalized talking-head synthesis
Huang, X., Li, X., Tan, S., Chen, W., & Li, G. (2025) · The Visual Computer · doi:10.1007/s00371-025-04195-y · Cited by 3
Cui, J., Chen, B., Xu, M., Shang, H., Chen, Y., Su, Q., Dong, Z., Yao, Y., Wang, J., & Zhu, S. (2025) · SIGGRAPH Asia Conference Papers · doi:10.1145/3757377.3763914 · Cited by 3
GaussianSpeech: Audio-Driven Personalized 3D Gaussian Avatars
Aneja, S., Sevastopolsky, A., Kirschstein, T., Thies, J., Dai, A., & Nießner, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01214 · Cited by 3
Fan, X., Gao, H., Chen, Z., Chang, P., Han, M., & Hasegawa‐Johnson, M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00447 · Cited by 3
ConsistentAvatar: Learning to Diffuse Fully Consistent Talking Head Avatar with Temporal Guidance
Yang, H., Zhang, Z., Tang, H., Qian, J., & Yang, J. J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680619 · Cited by 6
Interactive Conversational Head Generation
Zhou, M., Bai, Y., Zhang, W., Yao, T., & Zhao, T. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3562651 · Cited by 9
An approach to optimizing semantic consistency for text-to-digital human generation
Liu, X., Liu, X., Yang, P., Wang, Z., & Liu, F. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111909 · Cited by 4
Self-Guidance: Boosting Flow and Diffusion Generation on Their Own
Li, T., Luo, W., Chen, Z., Ma, L., & Qi, G.-J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3611831 · Cited by 2
Talking Face Generation via Face Mesh - Controllability without Reference Videos
Köksal, A., Xu, Q., & Lim, J. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00246 · Cited by 2
Multi-Level Feature Dynamic Fusion Neural Radiance Fields for Audio-Driven Talking Head Generation
Song, W., Liu, Q., Liu, Y., Zhang, P., & Cao, J. (2025) · Applied Sciences · doi:10.3390/app15010479 · Cited by 2
Adaptive diffusion landmark dynamic rendering for realistic talking face video generation
Tang, Y., Yazhi, L., Xiong, L., & Li, W. (2025) · The Visual Computer · doi:10.1007/s00371-025-03907-8 · Cited by 4
Self-citationFastTalker: Real-time audio-driven talking face generation with 3D Gaussian
Chen, K., Li, Z., Cui, F., Ni, M., Wang, S., Che, J., Liu, F., Qi, Y., Zhang, F., Liu, J., et al. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105573 · Cited by 2
Oskooei, A. R., Caglar, E., Şahin, I., Kayabay, A., & Aktaş, M. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97606-3_15 · Cited by 2
Text-Driven Talking Face Synthesis by Reprogramming Audio-Driven Models
Choi, J., Kim, M., Park, S. J., & Ro, Y. M. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448506 · Cited by 1
Identity-Preserving Audio-Driven Holistic Human Motion Video Generation
Xue, H., Zhang, Z., Li, M., Dai, Z., & Wu, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890615 · Cited by 1
Expressive talking face generation via audio visual control
Li, P., Zhao, H., Lin, M., Liu, Q., Tang, P., & Zhou, Y. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01759-9 · Cited by 1
Talking Face Generation With Lip and Identity Priors
Wu, J., Li, F. W. B., Tam, G. K. L., Yang, B., Nan, F., & Pan, J. (2025) · Computer Animation and Virtual Worlds · doi:10.1002/cav.70026 · Cited by 1
Dai, Q., Feng, H., Cui, W., Cai, X., Zheng, Y., & Zeng, M. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733322 · Cited by 1
Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning
Xie, Y., Ma, F., Bin, Y., He, Y., & Yu, F. R. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733279 · Cited by 3
Generative AI for Video Translation: A Scalable Architecture for Multilingual Video Conferencing
Oskooei, A. R., Caglar, E., Şahin, I., Kayabay, A., & Aktaş, M. S. (2025) · Applied Sciences · doi:10.3390/app152312691 · Cited by 1
A Survey of Talking Head Synthesis Techniques: Portrait Generation, Driving Mechanisms, and Editing
Meng, M., Zhao, Y., Zhang, B., Zhu, Y., Shi, W., Wen, Z., & Fan, Z. (2025) · ACM Computing Surveys · doi:10.1145/3785656 · Cited by 8
Text Image Generation for Low-Resource Languages with Dual Translation Learning
Noguchi, C., Fukuda, S., Mihara, S., & Yamanaka, M. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00772 · Cited by 1
Shi, L., Jiang, B., Yuan, Z., Pérez, M. A., & Guo, F. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00482 · Cited by 1
DH-FaceVid-1K: A Large-Scale High-Quality Dataset for Face Video Generation
Di, D., He, F., Sun, W., Ma, Y., Li, H., Chen, W., Fan, L., Su, T., & Yang, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01127 · Cited by 7
Yee, P. T., Kollias, D., Mishra, S., & Dhall, A. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00442 · Cited by 1
Learning Disentangled Speech- and Expression-Driven Blendshapes for 3D Face Animation
Mao, Y., Zhang, Z., Zhang, Z., Liu, J., Zeng, C., & Xia, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7195-6_14 · Cited by 1
Zhao, G., Liu, Y., Wang, X., & Yan, F. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658055 · Cited by 1
3D-Aware Text-Driven Talking Avatar Generation
Wu, X., Sun, Y.-T., Chen, H., Zhou, H., Wang, J., Liu, Z., & Qi, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73223-2_23 · Cited by 1
Neural Fields for Multimedia Applications (Part-1): Trends in Audio, Images and Videos
Dutta, U. K., Choudhury, A., Muraleedharan, L. P., Liu, J., & Su, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82475-3_19 · Cited by 1
Joint Co-Speech Gesture and Expressive Talking Face Generation Using Diffusion with Adapters
Hogue, S., Zhang, C., Tian, Y., & Guo, X. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00409 · Cited by 1
A Microservice-Based Implementation of Chinese Conversational Digital Avatars Using NVIDIA ACE
Wu, F., Chen, K., & Dellinger, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-08557-3_9 · Cited by 1
DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model
Liu, K., Liu, J., Cao, Y., Guo, J., & Yi, X. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209940 · Cited by 1
High-Fidelity and Lip-Synced Talking Face Synthesis via Landmark-Based Diffusion Model
Zhong, W., Lin, J., Chen, P., Gao, F., Lin, L., & Li, G. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3676251 · Cited by 2
MoCha: Towards Movie-Grade Talking Character Synthesis
Wei, C., Sun, B., Ma, H., Hou, J., Juefei-Xu, F., He, Z., Dai, X., Zhang, L., Li, K., Hou, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.23307
3D-GS Talker: 3D Gaussian Based Audio-Driven Real-Time Talking Head GenerationPreprint
Chen, K., Huang, Y., & Liu, Z. (2024) · Research Square · doi:10.21203/rs.3.rs-4558717/v1
Embedded Representation Learning Network for Animating Styled Video Portrait
Wang, T., Liang, X., Zheng, W., Niu, D., Xia, H., & Xia, S. (2024) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg59268.2024.10582043
HBF Talk: Speech-Driven 3D Facial Animation Synthesis Research
王, 文. (2024) · Computer Science and Application · doi:10.12677/csa.2024.148174
Co-speech Gesture Video Generation with 3D Human Meshes
Mahapatra, A., Mishra, R., Li, R., Chen, Z., Ding, B., Wang, S., Zhu, J.-Y., Chang, P., Han, M., & Xiao, J. Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73024-5_11
StableTalk: Advancing Audio-to-Talking Face Generation with Stable Diffusion and Vision Transformer
Nazarieh, F., Kittler, J., Awais, M., Kanojia, D., & Feng, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78172-8_18
A Lightweight and High-Fidelity Model for Generalized Audio-Driven 3D Talking Face Synthesis
Liu, S., Zhong, Y., Wang, H., & Peng, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78125-4_6
ECGNet-Potential Diffusion Model for Optimized and Robust Animated Character Generation
An, Z., & Wang, Y. (2025) · Journal of Circuits Systems and Computers · doi:10.1142/s0218126625502299
DFNeRF: Disentangled Facial Neural Radiance Fields for Text-based Editing of Free-view Talking Head
Chen, B., Li, X., Wang, X., Zhang, Q., & Wang, H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889463
One-Shot Talking Face Generation with Expression Editing
Huang, G., & Shen, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887606
High-fidelity Talking Face Synthesis Guided by Landmark
Liu, Z., Zhang, R., Li, B. H., & Liu, B. (2024) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic64248.2024.10913301 · Cited by 1
Metadata
- Authors
- Shen, S., Zhao, W., Meng, Z., Li, W., Zhu, Z., Zhou, J., & Lu, J.
- Affiliations
- Tsinghua University (CN)
- PhiGent Robotics
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 1982-1991
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386075487
- Cited by
- 133
- References
- 58
- Citations by year
- 2026: 25 · 2025: 56 · 2024: 43 · 2023: 9
- FWCI
- 14.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key Research and Development Program of China (2017YFA0700802)
- National Natural Science Foundation of China (62125603)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- talking head synthesis, audio-driven video generation, latent diffusion models, facial landmarks, audio-video synchronization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
