DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation
Open accessAbstract
Talking head synthesis is a promising approach for the video production industry. Recently, a lot of effort has been devoted in this research area to improve the generation quality or enhance the model generalization. However, there are few works able to address both issues simultaneously, which is essential for practical applications. To this end, in this paper, we turn attention to the emerging powerful Latent Diffusion Models, and model the Talking head generation as an audio-driven temporally coherent denoising process (DiffTalk). More specifically, instead of employing audio signals as the single driving factor, we investigate the control mechanism of the talking face, and incorporate reference face images and landmarks as conditions for personality-aware generalized synthesis. In this way, the proposed DiffTalk is capable of producing high-quality talking head videos in synchronization with the source audio, and more importantly, it can be naturally generalized across different identities without further finetuning. Additionally, our DiffTalk can be gracefully tailored for higher-resolution synthesis with negligible extra computational cost. Extensive experiments show that the proposed DiffTalk efficiently synthesizes high-fidelity audio-driven talking head videos for generalized novel identities. For more video results, please refer to https://sstzal.github.io/DiffTalk/.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several face reenactment strategies including EAMM, AVFR-GAN, DiffTalk, ADNeRF and ATVGnet are not well suited for zero-shot lip-synced generation of unseen speakers.
“We investigated several face reenactment strategies including EAMM [26], AVFR-GAN [2], DiffTalk [54], ADNeRF [20] and ATVGnet [10] and concluded that these methods are not well suited for zero-shot lip-synced generation of unseen speakers.”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [54]
Pioneer talking-head generation works typically utilize biometric signals such as facial keypoints or 3D priors as intermediate motion representation to animate the reference face while ensuring lip synchronization.
“pioneer talking-head generation works typically utilize biometric signals such as facial keypoints [22, 28, 30, 45], or 3D priors [11, 13, 16, 23, 32, 43, 44] as intermediate motion representation to animate the reference face while ensuring lip synchronization.”
From MoCha: Towards Movie-Grade Talking Character Synthesis (2025) · cites this paper as [30]
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · see citing works
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · see citing works
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351 · see citing works
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362 · see citing works
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637 · see citing works
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514 · see citing works
Denoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507 · see citing works
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · see citing works
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., & Aberman, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02155 · Cited by 2,295 · see citing works
Practical markov chain monte carlo
Geyer, C. J. (1992) · Statistical Science · doi:10.1214/ss/1177011137 · Cited by 2,045 · see citing works
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778 · see citing works
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747 · see citing works
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057 · see citing works
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897 · see citing works
Imagic: Text-Based Real Image Editing with Diffusion Models
Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., & Irani, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00582 · Cited by 813 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737 · see citing works
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
Chan, E. R., Monteiro, M., Kellnhofer, P., Wu, J., & Wetzstein, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00574 · Cited by 697 · see citing works
Out of Time: Automated Lip Sync in the Wild
Chung, J. S., & Zisserman, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54427-4_19 · Cited by 689 · see citing works
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687 · see citing works
MotionDiffuse: Text-Driven Human Motion Generation With Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355414 · Cited by 566 · see citing works
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
Wang, T.-C., Mallya, A., & Liu, M.-Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00991 · Cited by 468 · see citing works
Hierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss
Chen, L., Maddox, R. K., Duan, Z., & Xu, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00802 · Cited by 437 · see citing works
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Zhang, M., Cai, Z., Pan, L., Hong, F., Guo, X., Yang, L., & Liu, Z. (2022) · arXiv · doi:10.48550/arxiv.2208.15001 · Cited by 437 · see citing works
Zhou, Y., Han, X., Shechtman, E., Echevarria, J., Kalogerakis, E., & Li, D. (2020) · ACM Transactions on Graphics · doi:10.1145/3414685.3417774 · Cited by 403 · see citing works
Neural Voice Puppetry: Audio-Driven Facial Reenactment
Thies, J., Elgharib, M. A., Tewari, A., Theobalt, C., & Nießner, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_42 · Cited by 397 · see citing works
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
Guo, Y., Chen, K., Liang, S., Liu, Y.-J., Bao, H., & Zhang, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00573 · Cited by 384 · see citing works
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
Zhou, H., Sun, Y., Wu, W., Loy, C. C., Wang, X., & Liu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00416 · Cited by 377 · see citing works
Capture, Learning, and Synthesis of 3D Speaking Styles
Cudeiro, D., Bolkart, T., Laidlaw, C., Ranjan, A., & Black, M. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01034 · Cited by 371 · see citing works
Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset
Zhang, Z., Li, L., Ding, Y., & Fan, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00366 · Cited by 368 · see citing works
A Morphable Model For The Synthesis Of 3D Faces
Blanz, V., & Vetter, T. (2023) · ACM eBooks · doi:10.1145/3596711.3596730 · Cited by 364 · see citing works
Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Huang, Z., Zhang, T., Heng, W., Shi, B., & Zhou, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_36 · Cited by 341 · see citing works
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications
Zollhöfer, M., Thies, J., Garrido, P., Bradley, D., Beeler, T., Pérez, P., Stamminger, M., Nießner, M., & Theobalt, C. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13382 · Cited by 307 · see citing works
Diffusion Posterior Sampling for General Noisy Inverse ProblemsPreprint
Chung, H., Kim, J., McCann, M. T., Klasky, M., & Ye, J. C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.14687 · Cited by 293 · see citing works
Audio-Driven Emotional Video Portraits
Ji, X., Zhou, H., Wang, K., Wu, W., Loy, C. C., Cao, X., & Xu, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01386 · Cited by 223 · see citing works
Talking-Head Generation with Rhythmic Head Motion
Chen, L., Cui, G., Liu, C., Li, Z., Kou, Z., Xu, Y., & Xu, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_3 · Cited by 179 · see citing works
Towards Automatic Face-to-Face Translation
R, P. K., Mukhopadhyay, R., Philip, J., Jha, A., Namboodiri, V. P., & Jawahar, C. V. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351066 · Cited by 151 · see citing works
Everybody’s Talkin’: Let Me Talk as You Want
Song, L., Wu, W., Qian, C., He, R., & Loy, C. C. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3146783 · Cited by 142 · see citing works
VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track
Garrido, P., Valgaerts, L., Sarmadi, H., Steiner, I., Varanasi, K., Pérez, P., & Theobalt, C. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12552 · Cited by 127 · see citing works
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation
Liu, X., Xu, Y., Wu, Q., Zhou, H., Wu, W., & Zhou, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_7 · Cited by 121 · see citing works
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis
Shen, S., Li, W., Zhu, Z. D., Duan, Y., Zhou, J., & Lu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_39 · Cited by 107 · see citing works
Self-citationSpeech-Driven Facial Animation Using Cascaded GANs for Learning of Motion and Texture
Das, D., Biswas, S., Sinha, S., & Bhowmick, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_25 · Cited by 101 · see citing works
GAUDI: A Neural Architect for Immersive 3D Scene Generation
Bautista, M. Á., Guo, P., Abnar, S., Talbott, W., Toshev, A., Chen, Z., Dinh, L., Zhai, S., Goh, H., Ulbricht, D., et al. (2022) · neural information processing systems · doi:10.52202/068431-1820 · Cited by 58 · see citing works
FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesis
Gu, K., Zhou, Y., & Huang, T. S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6717 · Cited by 55 · see citing works
DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural RenderingPreprint
Yao, S., Zhong, R., Yan, Y., Zhai, G., & Yang, X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.00791 · Cited by 55 · see citing works
DiffuStereo: High Quality Human Reconstruction via Diffusion-Based Stereo Using Sparse Cameras
Shao, R., Zheng, Z., Zhang, H., Sun, J., & Liu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_41 · Cited by 53 · see citing works
DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional LatentsPreprint
Pandey, K., Mukherjee, A., Rai, P., & Kumar, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.00308 · Cited by 51 · see citing works
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion ModelsPreprint
Rombach, R., Blattmann, A., & Ommer, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.13038 · Cited by 43 · see citing works
A generic diffusion-based approach for 3D human pose prediction in the wild
Saadatnejad, S., Rasekh, A., Mofayezi, M., Medghalchi, Y., Rajabzadeh, S., Mordan, T., & Alahi, A. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160399 · Cited by 39 · see citing works
A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images
Waibel, D., Röell, E., Rieck, B., Giryes, R., & Marr, C. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230752 · Cited by 35 · see citing works
DAVD-Net: Deep Audio-Aided Video Decompression of Talking Heads
Zhang, X., Wu, X., Zhai, X., Ben, X., & Tu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01235 · Cited by 29 · see citing works
Face Super-Resolution Using Stochastic Differential Equations
dos Santos, M., Laroca, R., Ribeiro, R. O., Neves, J. C., Proença, H. P. M. C., & Menotti, D. (2022) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi55357.2022.9991799 · Cited by 13 · see citing works
HeadGAN: Video-and-Audio-Driven Talking Head SynthesisPreprint
Doukas, M. C., Zafeiriou, S. P., & Sharmanska, V. (2020) · arXiv (Cornell University) · Cited by 6 · see citing works
Intelligent Painter: Picture Composition with Resampling Diffusion Model
Ku, W.-F., Siu, W.-C., Cheng, X., & Chan, H. A. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222829 · Cited by 5 · see citing works
(2014) · doi:10.4230/oasics.slate.2021.16 · Cited by 1 · see citing works
Flagged first, then by citations
1 of the 58 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 162 · see citing works
State of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 157 · see citing works
Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis
Li, J., Zhang, J., Bai, X. P., Zhou, J. W., & Gu, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00696 · Cited by 116 · see citing works
Self-citationFace Generation and Editing With StyleGAN: A Survey
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3350004 · Cited by 105 · see citing works
Tian, L., Wang, Q. J., Zhang, B., & Bo, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73010-8_15 · Cited by 112 · see citing works
Tian, L., Wang, Q., Zhang, B., & Bo, L. (2024) · arXiv · doi:10.48550/arxiv.2402.17485 · Cited by 92 · see citing works
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Cai, Z., Ghosh, S., Adatia, A. P., Hayat, M., Dhall, A., Gedeon, T., & Stefanov, K. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3680795 · Cited by 59 · see citing works
“The variation in the number of subjects and videos in different sets is presented in Table 4 and Figure 4.”
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models
Aneja, S., Thies, J., Dai, A., & NieBner, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02009 · Cited by 55 · see citing works
EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis
Tan, S., Ji, B., Bi, M., & Pan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_23 · Cited by 45 · see citing works
SD-NeRF: Towards Lifelike Talking Head Animation via Spatially-Adaptive Dual-Driven NeRFs
Shen, S., Li, W., Huang, X., Zhu, Z. D., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3308441 · Cited by 39 · see citing works
Self-citationSpeech driven video editing via an audio-conditioned diffusion model
Bigioi, D., Basak, S., Stypułkowski, M., Zięba, M., Jordan, H., McDonnell, R., & Corcoran, P. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.104911 · Cited by 42 · see citing works
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization
Mukhopadhyay, S., Suri, S., Gadde, R. T., & Shrivastava, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00521 · Cited by 42 · see citing works
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cui, J., Li, H., Zhan, Y., Shang, H. L., Cheng, K., Ma, Y., Mu, S., Zhou, H., Wang, J., & Zhu, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01964 · Cited by 34 · see citing works
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cui, J., Li, H., Zhan, Y., Shang, H., Cheng, K., Ma, Y., Mu, S., Zhou, H., Wang, J., & Zhu, S. (2024) · arXiv · doi:10.48550/arxiv.2412.00733 · Cited by 32 · see citing works
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 29 · see citing works
PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models
Zhang, Y., Xing, Z., Zeng, Y., Fang, Y., & Chen, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00740 · Cited by 25 · see citing works
Li, J., Zhang, J., Bai, X. P., Zheng, J., Zhou, J. W., & Gu, L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102456 · Cited by 22 · see citing works
Self-citationHuman Motion Video Generation: A Survey
Xue, H., Luo, X., Hu, Z., Zhang, X., Xiang, X., Dai, Y., Liu, J., Zhang, Z., Li, M., Yang, J. J., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3594034 · Cited by 24 · see citing works
D 4 M: Dataset Distillation via Disentangled Diffusion Model
Su, D., Hou, J., Gao, W., Tian, Y., & Tang, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00555 · Cited by 21 · see citing works
DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generation
Hong, F.-T., Shen, L., & Xu, D. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339964 · Cited by 22 · see citing works
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions
Liu, Y., Lin, L., Yu, F., Zhou, C., & Li, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02104 · Cited by 20 · see citing works
AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding
Liu, T., Chen, F., Fan, S., Du, C., Chen, Q., Chen, X., & Yu, K. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681198 · Cited by 21 · see citing works
DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction
Xiong, J., Zhang, P., You, T., Li, C., Huang, W., & Zha, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02575 · Cited by 19 · see citing works
Application of a 3D Talking Head as Part of Telecommunication AR, VR, MR System: Systematic Review
Christoff, N., Neshov, N. N., Tonchev, K., & Manolova, A. H. (2023) · Electronics · doi:10.3390/electronics12234788 · Cited by 17 · see citing works
Audio-Driven Facial Animation with Deep Learning: A Survey
Jiang, D., Chang, J. M., You, L., Bian, S., Kosk, R., & Maguire, G. (2024) · Information · doi:10.3390/info15110675 · Cited by 20 · see citing works
LaMD: Latent Motion Diffusion for Image-Conditional Video Generation
Hu, Y., Chen, Z., & Luo, C. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02386-7 · Cited by 24 · see citing works
Wav2Lip‐HR: Synthesising clear high‐resolution talking head in the wild
Liang, C., Wang, Q., Chen, Y., & Tang, M. (2023) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2226 · Cited by 15 · see citing works
Sonic: Shifting Focus to Global Audio Perception in Portrait Animation
Ji, X., Hu, X., Xu, Z., Zhu, J., Lin, C., He, Q., Zhang, J., Luo, D., Chen, Y., Lin, Q., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00027 · Cited by 21 · see citing works
Revisiting Generalizability in Deepfake Detection: Improving Metrics and Stabilizing Transfer
Kamat, S., Agarwal, S., Darrell, T., & Rohrbach, A. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00049 · Cited by 15 · see citing works
EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model
Zhang, B., Zhang, X., Cheng, N., Yu, J., Xiao, J. Z., & Wang, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447505 · Cited by 13 · see citing works
AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis
Li, D., Zhao, K., Wang, W., Peng, B., Zhang, Y., Dong, J., & Tan, T. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28086 · Cited by 13 · see citing works
Increasing Importance of Joint Analysis of Audio and Video in Computer Vision: A Survey
Shahabaz, A., & Sarkar, S. (2024) · IEEE Access · doi:10.1109/access.2024.3391817 · Cited by 13 · see citing works
Seeing the Sound: Multilingual Lip Sync for Real-Time Face-to-Face Translation
Oskooei, A. R., Aktaş, M. S., & Keleş, M. (2024) · Computers · doi:10.3390/computers14010007 · Cited by 13 · see citing works
Audio-Semantic Enhanced Pose-Driven Talking Head Generation
Liu, M., Li, D., Li, Y., Song, X., & Nie, L. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3414412 · Cited by 11 · see citing works
3D facial modeling, animation, and rendering for digital humans: A survey
Zhang, Y., Su, R., Yu, J., & Li, R. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128168 · Cited by 10 · see citing works
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing
Xiong, L., Cheng, X., Tan, J., Wu, X., Li, X., Zhu, L., Ma, F., Li, M., Xu, H., & Hu, Z. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681108 · Cited by 10 · see citing works
SingingHead: A Large-Scale 4D Dataset for Singing Head Animation
Wu, S., Li, Y., Zhang, W., Jia, J., Zhu, Y., Yan, Y., Zhai, G., & Yang, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3623560 · Cited by 10 · see citing works
PersonaTalk: Bring Attention to Your Persona in Visual Dubbing
Zhang, L., Liang, S., Ge, Z., & Hu, T. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687618 · Cited by 9 · see citing works
Hierarchical Feature Warping and Blending for Talking Head Animation
Zhang, J., Liu, C., Xian, K., & Cao, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3375330 · Cited by 9 · see citing works
DiffDub: Person-Generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-Encoder
Liu, T., Du, C., Fan, S., Chen, F., & Yu, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446049 · Cited by 8 · see citing works
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment
Bounareli, S., Tzelepis, C., Argyriou, V., Patras, I., & Tzimiropoulos, G. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099159 · Cited by 14 · see citing works
Multimodal Fusion for Talking Face Generation Utilizing Speech-Related Facial Action Units
Liu, Z., Liu, X., Chen, S., Liu, J., Wang, L., & Bi, C. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3672565 · Cited by 7 · see citing works
Audio-Driven Talking Face Generation with Stabilized Synchronization Loss
Yaman, D., Eyiokur, F. İ., Bärmann, L., Ekenel, H. K., & Waibel, A. H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72655-2_24 · Cited by 7 · see citing works
Multimodal Feature-Guided Audio-Driven Emotional Talking Face Generation
Wang, X., Huo, Y., Liu, Y., Guo, X., Yan, F., & Zhao, G. (2025) · Electronics · doi:10.3390/electronics14132684 · Cited by 8 · see citing works
Decoupled Two-Stage Talking Head Generation via Gaussian-Landmark-Based Neural Radiance Fields
Ma, B., Cao, Y., & Zhang, L. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450482 · Cited by 7 · see citing works
ReSyncer: Rewiring Style-Based Generator for Unified Audio-Visually Synced Facial Performer
Guan, J., Xu, Z., Zhou, H., Wang, K., He, S., Zhang, Z., Liang, B., Feng, H., Ding, E., Liu, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72940-9_20 · Cited by 7 · see citing works
Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generation
Ye, Z., Chen, Z., Li, T., Huang, Z., Luo, W., & Qi, G.-J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02180 · Cited by 6 · see citing works
VIIS: Visible and Infrared Information Synthesis for Severe Low-Light Image Enhancement
Zhao, C., Yu, M., Yang, F., & Jing, P. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00218 · Cited by 6 · see citing works
DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis
Deng, Y., Zheng, D., Xie, J., Wang, J., Xie, W., Shen, L., & Song, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890278 · Cited by 5 · see citing works
FD2Talk: Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model
Yao, Z., Cheng, X., & Huang, Z. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681238 · Cited by 5 · see citing works
Rakesh, V. K., Mazumdar, S., Pal, S., Das, A. K., & Samanta, T. (2025) · The Visual Computer · doi:10.1007/s00371-025-04232-w · Cited by 5 · see citing works
Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation
Tan, J., Cheng, X., Xiong, L., Zhu, L., Li, X., Wu, X., Gong, K., Li, M., & Cai, Y. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10688218 · Cited by 4 · see citing works
Zhang, C., Wang, C., Zhang, J., Xu, H., Song, G., Xie, Y., Luo, L., Tian, Y., Feng, J., & Guo, X. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450491 · Cited by 4 · see citing works
Learning and Evaluating Human Preferences for Conversational Head Generation
Zhou, M., Bai, Y., Zhang, W., Yao, T., Zhao, T., & Mei, T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612831 · Cited by 4 · see citing works
Gao, Y., Li, J., Chu, L., & Lü, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01824 · Cited by 4 · see citing works
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance
Zhang, H., Yuan, Z., Zheng, C., Yan, X., Wang, B., Li, G., Wu, S., Cui, S. R., & Li, Z. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3566382 · Cited by 3 · see citing works
GaPTalk: precision-controlled 3D Gaussian rendering for personalized talking-head synthesis
Huang, X., Li, X., Tan, S., Chen, W., & Li, G. (2025) · The Visual Computer · doi:10.1007/s00371-025-04195-y · Cited by 3 · see citing works
Cui, J., Chen, B., Xu, M., Shang, H., Chen, Y., Su, Q., Dong, Z., Yao, Y., Wang, J., & Zhu, S. (2025) · SIGGRAPH Asia Conference Papers · doi:10.1145/3757377.3763914 · Cited by 3 · see citing works
GaussianSpeech: Audio-Driven Personalized 3D Gaussian Avatars
Aneja, S., Sevastopolsky, A., Kirschstein, T., Thies, J., Dai, A., & Nießner, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01214 · Cited by 3 · see citing works
Fan, X., Gao, H., Chen, Z., Chang, P., Han, M., & Hasegawa‐Johnson, M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00447 · Cited by 3 · see citing works
ConsistentAvatar: Learning to Diffuse Fully Consistent Talking Head Avatar with Temporal Guidance
Yang, H., Zhang, Z., Tang, H., Qian, J., & Yang, J. J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680619 · Cited by 3 · see citing works
Interactive Conversational Head Generation
Zhou, M., Bai, Y., Zhang, W., Yao, T., & Zhao, T. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3562651 · Cited by 9 · see citing works
An approach to optimizing semantic consistency for text-to-digital human generation
Liu, X., Liu, X., Yang, P., Wang, Z., & Liu, F. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111909 · Cited by 2 · see citing works
Self-Guidance: Boosting Flow and Diffusion Generation on Their Own
Li, T., Luo, W., Chen, Z., Ma, L., & Qi, G.-J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3611831 · Cited by 2 · see citing works
Talking Face Generation via Face Mesh - Controllability without Reference Videos
Köksal, A., Xu, Q., & Lim, J. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00246 · Cited by 2 · see citing works
Multi-Level Feature Dynamic Fusion Neural Radiance Fields for Audio-Driven Talking Head Generation
Song, W., Liu, Q., Liu, Y., Zhang, P., & Cao, J. (2025) · Applied Sciences · doi:10.3390/app15010479 · Cited by 2 · see citing works
Adaptive diffusion landmark dynamic rendering for realistic talking face video generation
Tang, Y., Yazhi, L., Xiong, L., & Li, W. (2025) · The Visual Computer · doi:10.1007/s00371-025-03907-8 · Cited by 2 · see citing works
Self-citationFastTalker: Real-time audio-driven talking face generation with 3D Gaussian
Chen, K., Li, Z., Cui, F., Ni, M., Wang, S., Che, J., Liu, F., Qi, Y., Zhang, F., Liu, J., et al. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105573 · Cited by 2 · see citing works
Oskooei, A. R., Caglar, E., Şahin, I., Kayabay, A., & Aktaş, M. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97606-3_15 · Cited by 2 · see citing works
Text-Driven Talking Face Synthesis by Reprogramming Audio-Driven Models
Choi, J., Kim, M., Park, S. J., & Ro, Y. M. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448506 · Cited by 1 · see citing works
Identity-Preserving Audio-Driven Holistic Human Motion Video Generation
Xue, H., Zhang, Z., Li, M., Dai, Z., & Wu, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890615 · Cited by 1 · see citing works
Expressive talking face generation via audio visual control
Li, P., Zhao, H., Lin, M., Liu, Q., Tang, P., & Zhou, Y. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01759-9 · Cited by 1 · see citing works
Talking Face Generation With Lip and Identity Priors
Wu, J., Li, F. W. B., Tam, G. K. L., Yang, B., Nan, F., & Pan, J. (2025) · Computer Animation and Virtual Worlds · doi:10.1002/cav.70026 · Cited by 1 · see citing works
Dai, Q., Feng, H., Cui, W., Cai, X., Zheng, Y., & Zeng, M. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733322 · Cited by 1 · see citing works
Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning
Xie, Y., Ma, F., Bin, Y., He, Y., & Yu, F. R. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733279 · Cited by 2 · see citing works
Generative AI for Video Translation: A Scalable Architecture for Multilingual Video Conferencing
Oskooei, A. R., Caglar, E., Şahin, I., Kayabay, A., & Aktaş, M. S. (2025) · Applied Sciences · doi:10.3390/app152312691 · Cited by 1 · see citing works
A Survey of Talking Head Synthesis Techniques: Portrait Generation, Driving Mechanisms, and Editing
Meng, M., Zhao, Y., Zhang, B., Zhu, Y., Shi, W., Wen, Z., & Fan, Z. (2025) · ACM Computing Surveys · doi:10.1145/3785656 · Cited by 4 · see citing works
Text Image Generation for Low-Resource Languages with Dual Translation Learning
Noguchi, C., Fukuda, S., Mihara, S., & Yamanaka, M. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00772 · Cited by 1 · see citing works
Shi, L., Jiang, B., Yuan, Z., Pérez, M. A., & Guo, F. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00482 · Cited by 1 · see citing works
DH-FaceVid-1K: A Large-Scale High-Quality Dataset for Face Video Generation
Di, D., He, F., Sun, W., Ma, Y., Li, H., Chen, W., Fan, L., Su, T., & Yang, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01127 · Cited by 1 · see citing works
Yee, P. T., Kollias, D., Mishra, S., & Dhall, A. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00442 · Cited by 1 · see citing works
Learning Disentangled Speech- and Expression-Driven Blendshapes for 3D Face Animation
Mao, Y., Zhang, Z., Zhang, Z., Liu, J., Zeng, C., & Xia, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7195-6_14 · Cited by 1 · see citing works
Zhao, G., Liu, Y., Wang, X., & Yan, F. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658055 · Cited by 1 · see citing works
3D-Aware Text-Driven Talking Avatar Generation
Wu, X., Sun, Y.-T., Chen, H., Zhou, H., Wang, J., Liu, Z., & Qi, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73223-2_23 · Cited by 1 · see citing works
Neural Fields for Multimedia Applications (Part-1): Trends in Audio, Images and Videos
Dutta, U. K., Choudhury, A., Muraleedharan, L. P., Liu, J., & Su, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82475-3_19 · Cited by 1 · see citing works
Joint Co-Speech Gesture and Expressive Talking Face Generation Using Diffusion with Adapters
Hogue, S., Zhang, C., Tian, Y., & Guo, X. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00409 · Cited by 1 · see citing works
A Microservice-Based Implementation of Chinese Conversational Digital Avatars Using NVIDIA ACE
Wu, F., Chen, K., & Dellinger, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-08557-3_9 · Cited by 1 · see citing works
DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model
Liu, K., Liu, J., Cao, Y., Guo, J., & Yi, X. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209940 · Cited by 1 · see citing works
High-Fidelity and Lip-Synced Talking Face Synthesis via Landmark-Based Diffusion Model
Zhong, W., Lin, J., Chen, P., Gao, F., Lin, L., & Li, G. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3676251 · Cited by 2 · see citing works
MoCha: Towards Movie-Grade Talking Character Synthesis
Wei, C., Sun, B., Ma, H., Hou, J., Juefei-Xu, F., He, Z., Dai, X., Zhang, L., Li, K., Hou, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.23307
3D-GS Talker: 3D Gaussian Based Audio-Driven Real-Time Talking Head GenerationPreprint
Chen, K., Huang, Y., & Liu, Z. (2024) · Research Square · doi:10.21203/rs.3.rs-4558717/v1
Embedded Representation Learning Network for Animating Styled Video Portrait
Wang, T., Liang, X., Zheng, W., Niu, D., Xia, H., & Xia, S. (2024) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg59268.2024.10582043
HBF Talk: Speech-Driven 3D Facial Animation Synthesis Research
王, 文. (2024) · Computer Science and Application · doi:10.12677/csa.2024.148174
Co-speech Gesture Video Generation with 3D Human Meshes
Mahapatra, A., Mishra, R., Li, R., Chen, Z., Ding, B., Wang, S., Zhu, J.-Y., Chang, P., Han, M., & Xiao, J. Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73024-5_11
StableTalk: Advancing Audio-to-Talking Face Generation with Stable Diffusion and Vision Transformer
Nazarieh, F., Kittler, J., Awais, M., Kanojia, D., & Feng, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78172-8_18
A Lightweight and High-Fidelity Model for Generalized Audio-Driven 3D Talking Face Synthesis
Liu, S., Zhong, Y., Wang, H., & Peng, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78125-4_6
ECGNet-Potential Diffusion Model for Optimized and Robust Animated Character Generation
An, Z., & Wang, Y. (2025) · Journal of Circuits Systems and Computers · doi:10.1142/s0218126625502299
DFNeRF: Disentangled Facial Neural Radiance Fields for Text-based Editing of Free-view Talking Head
Chen, B., Li, X., Wang, X., Zhang, Q., & Wang, H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889463
One-Shot Talking Face Generation with Expression Editing
Huang, G., & Shen, L. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887606
High-fidelity Talking Face Synthesis Guided by Landmark
Liu, Z., Zhang, R., Li, B. H., & Liu, B. (2024) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic64248.2024.10913301
- Authors
- Shen, S., Zhao, W., Meng, Z., Li, W., Zhu, Z., Zhou, J., & Lu, J.
- Affiliations
- Tsinghua University (CN)
- PhiGent Robotics
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 1982-1991
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386075487
- Cited by
- 133
- References
- 58
- Citations by year
- 2026: 25 · 2025: 56 · 2024: 43 · 2023: 9
- FWCI
- 14.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key Research and Development Program of China (2017YFA0700802)
- National Natural Science Foundation of China (62125603)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- talking head synthesis, audio-driven video generation, latent diffusion models, facial landmarks, audio-video synchronization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
