Text-based Editing of Talking-head Video
Open accessAbstract
Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e. no jump cuts). Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame. To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material. The annotated parameters corresponding to the selected segments are seamlessly stitched together and used to produce an intermediate video representation in which the lower half of the face is rendered with a parametric face model. Finally, a recurrent video generation network transforms this representation to a photorealistic video that matches the edited transcript. We demonstrate a large variety of edits, such as the addition, removal, and alteration of words, as well as convincing language translation and full sentence synthesis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In ForgeryNet, fake images are manipulated using 15 different manipulations while videos are manipulated using only 8 of them.
“The fake images are manipulated using 15 different manipulations while the videos are manipulated using only 8 of them [27,28,29,30,31,32,33,34,35,36].”
From On the Generalization of Deep Learning Models in Video Deepfake Detection (2023) · cites this paper as [36]
The Video Editing section describes text-based edits to talking-head videos, but does not discuss ForgeryNet or count its image and video manipulations. Searches for ForgeryNet and the claimed 15-versus-8 breakdown found no supporting passage in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · see citing works
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 10,387 · see citing works
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. R. (1999) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/311535.311556 · Cited by 4,695 · see citing works
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791 · see citing works
TIMIT Acoustic-Phonetic Continuous Speech Corpus
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Pallett, D. S., Dahlgren, N. L., Zue, V. W., & Fiscus, J. G. (2020) · Americanae (AECID Library) · doi:10.35111/17gk-bn40 · Cited by 2,509 · see citing works
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747 · see citing works
Self-citationDARPA TIMIT:: acoustic-phonetic continuous speech corpus CD-ROM, NIST speech disc 1-1.1
Garofolo, J. S., Lamel, L., Fisher, W. M., Fiscus, J. G., Pallett, D., & Dahlgren, N. L. (1993) · doi:10.6028/nist.ir.4930 · Cited by 1,533 · see citing works
Unit selection in a concatenative speech synthesis system using a large speech database
Hunt, A., & Black, A. (1996) · 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings · doi:10.1109/icassp.1996.541110 · Cited by 1,508 · see citing works
Statistical parametric speech synthesis
Zen, H., Tokuda, K., & Black, A. W. (2009) · Speech Communication · doi:10.1016/j.specom.2009.04.004 · Cited by 1,202 · see citing works
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057 · see citing works
Photographic Image Synthesis with Cascaded Refinement Networks
Chen, Q., & Koltun, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.168 · Cited by 887 · see citing works
Chan, C., Ginosar, S., Zhou, T., & Efros, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00603 · Cited by 737 · see citing works
Kim, H., Garrido, P., Tewari, A., Xu, W., Thies, J., Nießner, M., Pérez, P., Richardt, C., Zollhöfer, M., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201283 · Cited by 692 · see citing works
Self-citationVideo Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669 · see citing works
Speaker identification on the SCOTUS corpus
Yuan, J., & Liberman, M. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2935783 · Cited by 600 · see citing works
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Liu, G., Tao, A., Kautz, J., & Catanzaro, B. (2018) · arXiv · doi:10.48550/arxiv.1808.06601 · Cited by 594 · see citing works
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 585 · see citing works
Self-citationFace transfer with multilinear models
Vlasic, D., Brand, M., Pfister, H., & Popović, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073209 · Cited by 565 · see citing works
Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
Tran, A. T., Hassner, T., Masi, I., & Medioni, G. G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.163 · Cited by 503 · see citing works
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhöfer, M., Kim, H., Garrido, P., Bernard, F., Pérez, P., & Theobalt, C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.401 · Cited by 398 · see citing works
Self-citationLearning Detailed Face Reconstruction from a Single Image
Richardson, E., Sela, M., Or-El, R., & Kimmel, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.589 · Cited by 339 · see citing works
Recycle-GAN: Unsupervised Video Retargeting
Bansal, A., Ma, S., Ramanan, D., & Sheikh, Y. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_8 · Cited by 339 · see citing works
Large Scale 3D Morphable Models
Booth, J. G., Roussos, A., Ponniah, A. J. T., Dunaway, D. J., & Zafeiriou, S. P. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1009-7 · Cited by 330 · see citing works
Unsupervised Training for 3D Morphable Model Regression
Genova, K., Cole, F. H., Maschinot, A., Sarna, A., Vlasic, D., & Freeman, W. T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00874 · Cited by 311 · see citing works
Self-citation3D Face Reconstruction by Learning from Synthetic Data
Richardson, E., Sela, M., & Kimmel, R. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.56 · Cited by 309 · see citing works
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications
Zollhöfer, M., Thies, J., Garrido, P., Bradley, D., Beeler, T., Pérez, P., Stamminger, M., Nießner, M., & Theobalt, C. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13382 · Cited by 307 · see citing works
Self-citationTrainable videorealistic speech animation
Ezzat, T., Geiger, G., & Poggio, T. (2002) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/566570.566594 · Cited by 304 · see citing works
A deep learning approach for generalized speech animation
Taylor, S. L., Kim, T., Yue, Y., Mahler, M., Krahe, J., Rodriguez, A. G., Hodgins, J. K., & Matthews, I. A. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073699 · Cited by 303 · see citing works
Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation
Sela, M., Richardson, E., & Kimmel, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.175 · Cited by 284 · see citing works
End-to-End 3D Face Reconstruction with Deep Neural Networks
Dou, P., Shah, S. K., & Kakadiaris, I. A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.164 · Cited by 271 · see citing works
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz
Tewari, A., Zollhöfer, M., Garrido, P., Bernard, F., Kim, H., Pérez, P., & Theobalt, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00270 · Cited by 270 · see citing works
Self-citationReconstruction of Personalized 3D Face Rigs from Monocular Video
Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2890493 · Cited by 260 · see citing works
Self-citationDynamic 3D avatar creation from hand-held video input
Ichim, A.-E., Bouaziz, S., & Pauly, M. V. (2015) · ACM Transactions on Graphics · doi:10.1145/2766974 · Cited by 249 · see citing works
Expressive expression mapping with ratio images
Liu, Z., Shan, Y., & Zhang, Z. (2001) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/383259.383289 · Cited by 241 · see citing works
Zhou, Y., Xu, Z., Landreth, C., Kalogerakis, E., Maji, S., & Singh, K. S. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201292 · Cited by 234 · see citing works
Blanz, V., Scherbaum, K., Vetter, T. R., & Seidel, H. (2004) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2004.00799.x · Cited by 231 · see citing works
Real-time high-fidelity facial performance capture
Cao, C., Bradley, D., Zhou, K., & Beeler, T. (2015) · ACM Transactions on Graphics · doi:10.1145/2766943 · Cited by 217 · see citing works
CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images
Guo, Y., Zhang, J., Cai, J., Jiang, B., & Zheng, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2837742 · Cited by 211 · see citing works
Averbuch‐Elor, H., Cohen‐Or, D., Kopf, J., & Cohen, M. F. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130818 · Cited by 187 · see citing works
Edwards, P., Landreth, C., Fiume, E. L., & Singh, K. S. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925984 · Cited by 186 · see citing works
Garrido, P., Valgaerts, L., Rehmsen, O., Thormaehlen, T., Pérez, P., & Theobalt, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.537 · Cited by 177 · see citing works
Self-citationNagano, K., Seo, J., Xing, J., Wei, L., Li, Z., Saito, S., Agarwal, A., Fursund, J., & Li, H. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275075 · Cited by 165 · see citing works
Trainable videorealistic speech animation
Ezzat, T., Geiger, G., & Poggio, T. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566594 · Cited by 130 · see citing works
VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track
Garrido, P., Valgaerts, L., Sarmadi, H., Steiner, I., Varanasi, K., Pérez, P., & Theobalt, C. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12552 · Cited by 127 · see citing works
Self-citationAutomatic acquisition of high-fidelity facial performances using monocular videos
Shi, F., Wu, H., Tong, X., & Chai, J. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661290 · Cited by 126 · see citing works
Computational video editing for dialogue-driven scenes
Leake, M., Davis, A., Truong, A., & Agrawala, M. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073653 · Cited by 126 · see citing works
Self-citationWarp-guided GANs for single-photo facial animation
Geng, J., Shao, T., Zheng, Y., Weng, Y., & Zhou, K. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275043 · Cited by 122 · see citing works
A Hybrid Model for Identity Obfuscation by Face Replacement
Sun, Q., Tewari, A., Xu, W., Fritz, M., Theobalt, C., & Schiele, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_34 · Cited by 121 · see citing works
Self-citationPavel, A., Reed, C., Hartmann, B., & Agrawala, M. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647400 · Cited by 112 · see citing works
Self-citationMartin-Brualla, R., Pandey, R., Yang, S., Pidlypenskyi, P., Taylor, J., Valentin, J., Khamis, S., Davidson, P., Tkach, A., Lincoln, P., et al. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275099 · Cited by 106 · see citing works
Self-citationTools for placing cuts and transitions in interview video
Berthouzoz, F., Li, W., & Agrawala, M. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185563 · Cited by 100 · see citing works
Self-citationDriving High-Resolution Facial Scans with Video Performance Capture
Fyffe, G., Jones, A., Alexander, O., Ichikari, R., & Debevec, P. E. (2014) · ACM Transactions on Graphics · doi:10.1145/2638549 · Cited by 92 · see citing works
Realistic Dynamic Facial Textures from a Single Image Using GANs
Olszewski, K., Li, Z., Yang, C., Zhou, Y., Yu, R., Huang, Z., Xiang, S., Saito, S., Kohli, P., & Li, H. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.580 · Cited by 92 · see citing works
High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder
Tewari, A., Zollhöfer, M., Bernard, F., Garrido, P., Kim, H., Pérez, P., & Theobalt, C. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2876842 · Cited by 85 · see citing works
Self-citationTruong, A., Berthouzoz, F., Li, W., & Agrawala, M. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984511.2984569 · Cited by 82 · see citing works
Self-citationContent-based tools for editing audio stories
Rubin, S., Berthouzoz, F., Mysore, G. J., Li, W., & Agrawala, M. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2501993 · Cited by 75 · see citing works
Self-citationInternet Based Morphable Model
Kemelmacher-Shlizerman, I. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.404 · Cited by 70 · see citing works
Jin, Z., Mysore, G. J., DiVerdi, S., Lu, J., & Finkelstein, A. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073702 · Cited by 64 · see citing works
Self-citationTransferable videorealistic speech animation
Chang, Y., & Ezzat, T. (2005) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/1073368.1073388 · Cited by 51 · see citing works
Adaptive 3D Face Reconstruction from Unconstrained Photo Collections
Roth, J., Tong, Y., & Liu, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2636829 · Cited by 49 · see citing works
Pavel, A., Goldman, D. B., Hartmann, B., & Agrawala, M. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984511.2984552 · Cited by 43 · see citing works
Self-citationA Data-Driven Approach for Facial Expression Retargeting in Video
Li, K., Dai, Q., Wang, R., Liu, Y., Xu, F., & Wang, J. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2293064 · Cited by 37 · see citing works
Realistic facial expression synthesis for an image-based talking head
Liu, K., & Östermann, J. (2011) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2011.6011835 · Cited by 32 · see citing works
Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., & Seitz, S. M. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_25 · Cited by 23 · see citing works
Self-citationDynamic Authoring of Audio with Linked Scripts
Shin, H. V., Li, W., & Durand, F. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984511.2984561 · Cited by 21 · see citing works
Everybody Dance NowPreprint
Chan, C., Ginosar, S., Zhou, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07371 · Cited by 21 · see citing works
Neural Animation and Reenactment of Human Actor VideosPreprint
Liu, L., Xu, W., Zollhöfer, M., Kim, H., Bernard, F., Habermann, M., Wang, W., & Theobalt, C. (2018) · arXiv (Cornell University) · Cited by 17 · see citing works
Self-citationNeural Rendering and Reenactment of Human Actor VideosPreprint
Liu, L., Xu, W., Zollhöfer, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03658 · Cited by 10 · see citing works
Self-citationOptimized Photorealistic Audiovisual Speech Synthesis Using Active Appearance Modeling
Mattheyses, W., Latacz, L., & Verhelst, W. (2010) · VUBIR (Vrije Universiteit Brussel) · Cited by 9 · see citing works
Photorealistic 2D audiovisual text-to-speech synthesis using active appearance models
Mattheyses, W., & Verhelst, W. (2010) · doi:10.1145/1924035.1924042 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- [23]23. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems.
- [29]29. Zeyu Jin, Gautham J Mysore, Stephen Diverdi, Jingwan Lu, and Adam Finkelstein. 2017. VoCo: text-based insertion and replacement in audio narration. ACM Transactions on Graphics (TOG) 36, 4 (2017), 96.
- [30]30. Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations (ICLR).
- [34]34. H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, N. Nießner, P. Pérez, C. Richardt, M. Zollhöfer, and C. Theobalt. 2018b. Deep Video Portraits. ACM Transactions on Graphics 2018 (TOG) (2018).
- [41]41. Ricardo Martin-Brualla, Rohit Pandey, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Julien Valentin, Sameh Khamis, Philip Davidson, Anastasia Tkach, Peter Lincoln, Adarsh Kowdle, Christoph Rhemann, Dan B Goldman, Cem Keskin, Steve Seitz, Shahram Izadi, and Sean Fanello. 2018. LookinGood: Enhancing Performance Capture with Real-time Neural Re-rendering. ACM Trans. Graph. 37, 6, Article 255 (December 2018), 14 pages.
- [47]47. Amy Pavel, Dan B Goldman, Björn Hartmann, and Maneesh Agrawala. 2016. VidCrit: Video-based Asynchronous Video Review. In Proc. of UIST. ACM, 517-528.
- [67]67. Anh Truong, Floraine Berthouzoz, Wilmot Li, and Maneesh Agrawala. 2016. Quickcut: An interactive tool for editing narrated video. In Proc. of UIST. 497-507.
- [71]71. Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018b. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs. In CVPR.
- [72]72. O. Wiles, A.S. Koepke, and A. Zisserman. 2018. X2Face: A network for controlling face generation by using images, audio, and pose codes. In European Conference on Computer Vision.
- [75]75. Yang Zhou, Zhan Xu, Chris Landreth, Evangelos Kalogerakis, Subhransu Maji, and Karan Singh. 2018. Visemenet: Audio-driven Animator-centric Speech Animation. ACM Trans. Graph. 37, 4, Article 161 (July 2018), 161:1-161:10 pages.
Cited by
Sorted by citations
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237 · see citing works
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,114 · see citing works
Self-citationMedia Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979 · see citing works
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897 · see citing works
Park, K., Sinha, U., Hedman, P., Barron, J. T., Bouaziz, S., Goldman, D. B., Martin-Brualla, R., & Seitz, S. M. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480487 · Cited by 736 · see citing works
Self-citationMasood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543 · see citing works
Kietzmann, J., Lee, L. W., McCarthy, I. P., & Kietzmann, T. C. (2020) · Business Horizons · doi:10.1016/j.bushor.2019.11.006 · Cited by 504 · see citing works
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464 · see citing works
“Audio-to-Video (A2V) [68] and Text-to-Video (T2V) [142].”
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
Wang, T.-C., Mallya, A., & Liu, M.-Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00991 · Cited by 468 · see citing works
State of the Art on Neural Rendering
Tewari, A., Fried, O., Thies, J., Sitzmann, V., Lombardi, S., Sunkavalli, K., Martin-Brualla, R., Simon, T., Saragih, J., Nießner, M., et al. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14022 · Cited by 458 · see citing works
Self-citationAbdal, R., Zhu, P., Mitra, N. J., & Wonka, P. (2021) · ACM Transactions on Graphics · doi:10.1145/3447648 · Cited by 433 · see citing works
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
Guo, Y., Chen, K., Liang, S., Liu, Y.-J., Bao, H., & Zhang, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00573 · Cited by 384 · see citing works
Neural Voice Puppetry: Audio-Driven Facial Reenactment
Thies, J., Elgharib, M. A., Tewari, A., Theobalt, C., & Nießner, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_42 · Cited by 397 · see citing works
Self-citationFlow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset
Zhang, Z., Li, L., Ding, Y., & Fan, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00366 · Cited by 368 · see citing works
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields
Wang, C., Chai, M., He, M., Chen, D., & Liao, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00381 · Cited by 352 · see citing works
MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation
Wang, K., Wu, Q., Song, L., Yang, Z., Wu, W., Qian, C., He, R., Qiao, Y., & Loy, C. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_42 · Cited by 344 · see citing works
DeepFake Detection for Human Face Images and Videos: A Survey
Malik, A., Kuribayashi, M., Abdullahi, S. M., & Khan, A. N. (2022) · IEEE Access · doi:10.1109/access.2022.3151186 · Cited by 297 · see citing works
FaceFormer: Speech-Driven 3D Facial Animation with Transformers
Fan, Y., Lin, Z., Saito, J., Wang, W., & Komura, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01821 · Cited by 266 · see citing works
A Survey on Deepfake Video Detection
Yu, P., Xia, Z., Fei, J., & Lu, Y. (2021) · IET Biometrics · doi:10.1049/bme2.12031 · Cited by 253 · see citing works
Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
Liu, M.-Y., Huang, X., Yu, J., Wang, T.-C., & Mallya, A. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3049196 · Cited by 236 · see citing works
Detecting Deep-Fake Videos from Phoneme-Viseme Mismatches
Agarwal, S., Farid, H., Fried, O., & Agrawala, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00338 · Cited by 234 · see citing works
Self-citationAudio-Driven Emotional Video Portraits
Ji, X., Zhou, H., Wang, K., Wu, W., Loy, C. C., Cao, X., & Xu, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01386 · Cited by 223 · see citing works
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior
Xing, J., Xia, M., Zhang, Y., Cun, X., Wang, J., & Wong, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01229 · Cited by 197 · see citing works
Anticipating and Addressing the Ethical Implications of Deepfakes in the Context of Elections
Diakopoulos, N. A., & Johnson, D. K. (2019) · New Media & Society · doi:10.1177/1461444820925811 · Cited by 210 · see citing works
Anticipating and Addressing the Ethical Implications of Deepfakes in the Context of Elections
Diakopoulos, N., & Johnson, D. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3474183 · Cited by 194 · see citing works
Talking-Head Generation with Rhythmic Head Motion
Chen, L., Cui, G., Liu, C., Li, Z., Kou, Z., Xu, Y., & Xu, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_3 · Cited by 179 · see citing works
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
Yin, F., Zhang, Y., Cun, X., Cao, M., Fan, Y., Wang, X., Bai, Q., Wu, B., Wang, J., & Yang, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_6 · Cited by 180 · see citing works
Lu, Y., Chai, J., & Cao, X. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480484 · Cited by 174 · see citing works
Generative AI-Driven Semantic Communication Networks: Architecture, Technologies, and Applications
Liang, C., Du, H., Sun, Y., Niyato, D. T., Kang, J., Zhao, D., & Imran, M. A. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3435524 · Cited by 170 · see citing works
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion
Wang, S., Li, L., Ding, Y., Fan, C., & Yu, X. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/152 · Cited by 169 · see citing works
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model
Ji, X., Zhou, H., Wang, K., Wu, Q., Wu, W., Xu, F., & Cao, X. (2022) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings · doi:10.1145/3528233.3530745 · Cited by 155 · see citing works
ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
He, Y., Gan, B., Chen, S., Zhou, Y., Yin, G., Song, L., Sheng, L., Shao, J., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00434 · Cited by 161 · see citing works
A comprehensive overview of Deepfake: generation, detection, datasets, and opportunities
Seow, J. W., Lim, M. K., Phan, R. C. W., & Liu, J. K. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.135 · Cited by 149 · see citing works
Use of a Capsule Network to Detect Fake Images and VideosPreprint
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12467 · Cited by 163 · see citing works
Everybody’s Talkin’: Let Me Talk as You Want
Song, L., Wu, W., Qian, C., He, R., & Loy, C. C. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3146783 · Cited by 142 · see citing works
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, M. C., Nguyen, D. T., Nguyen, D. T., & Nahavandi, S. (2020) · Figshare · doi:10.6084/m9.figshare.12731039.v1 · Cited by 122 · see citing works
One-Shot Talking Face Generation from Single-Speaker Audio-Visual Correlation Learning
Wang, S., Li, L., Ding, Y., & Yu, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20154 · Cited by 115 · see citing works
Abdal, R., Zhu, P., Mitra, N., & Wonka, P. (2021) · UCL Discovery (University College London) · Cited by 111 · see citing works
Improving Generalization by Commonality Learning in Face Forgery Detection
Yu, P., Fei, J., Xia, Z., Zhou, Z., & Weng, J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3146781 · Cited by 115 · see citing works
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis
Shen, S., Li, W., Zhu, Z. D., Duan, Y., Zhou, J., & Lu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_39 · Cited by 107 · see citing works
CollaboVR: A Reconfigurable Framework for Creative Collaboration in Virtual Reality
He, Z., Du, R., & Perlin, K. (2020) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar50242.2020.00082 · Cited by 103 · see citing works
Lahiri, A., Kwatra, V., Frueh, C., Lewis, J., & Bregler, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00278 · Cited by 101 · see citing works
Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars
Sun, J., Wang, X., Wang, L., Li, X., Zhang, Y., Zhang, H., & Liu, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02011 · Cited by 98 · see citing works
Preventing DeepFake Attacks on Speaker Authentication by Dynamic Lip Movement Analysis
Yang, C.-Z., Ma, J., Wang, S., & Liew, A. W. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3045937 · Cited by 99 · see citing works
Creating, Using, Misusing, and Detecting Deep Fakes
Farid, H. (2022) · Journal of Online Trust and Safety · doi:10.54501/jots.v1i4.56 · Cited by 92 · see citing works
Rewriting a Deep Generative Model
Bau, D., Liu, S., Tongzhou, W., Zhu, J.-Y., & Torralba, A. B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_21 · Cited by 86 · see citing works
DurIAN: Duration Informed Attention Network For Multimodal SynthesisPreprint
Yu, C., Lu, H., Hu, N., Yu, M., Weng, C., Xu, K., Liu, P., Tuo, D., Kang, S., Lei, G., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01700 · Cited by 84 · see citing works
Photorealistic Audio-driven Video Portraits
Wen, X. L., Wang, M., Richardt, C., Chen, Z.-Y., & Hu, S. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3023573 · Cited by 81 · see citing works
StyleTalk: One-Shot Talking Head Generation with Controllable Speaking Styles
Ma, Y., Wang, S., Hu, Z., Fan, C., Lv, T., Ding, Y., Deng, Z., & Yu, X. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25280 · Cited by 79 · see citing works
HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance FieldsPreprint
Park, K., Sinha, U., Hedman, P., Barron, J. T., Bouaziz, S., Goldman, D. B., Martin-Brualla, R., & Seitz, S. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.13228 · Cited by 77 · see citing works
Self-citationPavel, A., Reyes, G., & Bigham, J. P. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415864 · Cited by 74 · see citing works
Tolosana, R., Romero-Tapiador, S., Vera-Rodríguez, R., González-Sosa, E., & Fiérrez, J. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104673 · Cited by 74 · see citing works
FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3155571 · Cited by 74 · see citing works
Human-Computer Interaction System: A Survey of Talking-Head Generation
Zhen, R., Song, W., He, Q., Cao, J., Shi, L., & Luo, J. (2023) · Electronics · doi:10.3390/electronics12010218 · Cited by 76 · see citing works
Privacy Intelligence: A Survey on Image Privacy in Online Social Networks
Liu, C., Zhu, T., Zhang, J., & Zhou, W. (2022) · ACM Computing Surveys · doi:10.1145/3547299 · Cited by 84 · see citing works
Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generation
Li, L., Wang, S., Zhang, Z., Ding, Y., Zheng, Y., Yu, X., & Fan, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16286 · Cited by 68 · see citing works
A literature review and perspectives in deepfakes: generation, detection, and applications
Dagar, D., & Vishwakarma, D. K. (2022) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-022-00241-w · Cited by 72 · see citing works
DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video
Zhang, Z., Hu, Z., Deng, W., Fan, C., Lv, T., & Ding, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25464 · Cited by 67 · see citing works
VirtualCube: An Immersive 3D Video Communication System
Zhang, Y., Yang, J., Liu, Z., Wang, R., Chen, G., Tong, X., & Guo, B. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3150512 · Cited by 66 · see citing works
LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing
Wang, B. K., Li, Y., Lv, Z., Xia, H., Xu, Y., & Sodhi, R. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645143 · Cited by 72 · see citing works
Liu, Y., Nadai, M. D., Cai, D., Li, H., Alameda-Pineda, X., Sebe, N., & Lepri, B. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413505 · Cited by 62 · see citing works
MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation
Zhang, B., Qi, C., Zhang, P., Zhang, B., Wu, H., Chen, D., Chen, Q., Wang, Y., & Wen, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02116 · Cited by 58 · see citing works
DPE: Disentanglement of Pose and Expression for General Video Portrait Editing
Pang, Y., Zhang, Y., Quan, W., Fan, Y., Cun, X., Shan, Y., & Dong-ming Yan, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00049 · Cited by 56 · see citing works
Audio-Driven Talking Face Video Generation With Dynamic Convolution Kernels
Ye, Z., Xia, M., Yi, R., Zhang, J., Lai, Y., Huang, X., Zhang, G., & Liu, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3142387 · Cited by 55 · see citing works
AVscript: Accessible Video Editing with Audio-Visual Scripts
Huh, M., Yang, S., Peng, Y.-H., Chen, X., Kim, Y., & Pavel, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581494 · Cited by 50 · see citing works
Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation
Ma, Y., Liu, H., Wang, H., Pan, H., He, Y., Yuan, J., Zeng, A., Cai, C., Shum, H., Liu, W., et al. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687587 · Cited by 53 · see citing works
Crosscast: Adding Visuals to Audio Travel Podcasts
Xia, H., Jacobs, J., & Agrawala, M. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415882 · Cited by 45 · see citing works
Self-citationCross-Forgery Analysis of Vision Transformers and CNNs for Deepfake Image Detection
Coccomini, D. A., Caldelli, R., Falchi, F., Gennaro, C., & Amato, G. (2022) · Proceedings of the 1st International Workshop on Multimedia AI against Disinformation · doi:10.1145/3512732.3533582 · Cited by 47 · see citing works
Learned Spatial Representations for Few-shot Talking-Head Synthesis
Meshry, M., Suri, S., Davis, L. S., & Shrivastava, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01357 · Cited by 44 · see citing works
The Ethical Implications of Generative Audio Models: A Systematic Literature Review
Barnett, J. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604686 · Cited by 47 · see citing works
Deep Person Generation: A Survey from the Perspective of Face, Pose, and Cloth Synthesis
Sha, T., Zhang, W., Shen, T., Li, Z., & Mei, T. (2022) · ACM Computing Surveys · doi:10.1145/3575656 · Cited by 42 · see citing works
SD-NeRF: Towards Lifelike Talking Head Animation via Spatially-Adaptive Dual-Driven NeRFs
Shen, S., Li, W., Huang, X., Zhu, Z. D., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3308441 · Cited by 39 · see citing works
Age of Incorrect Information in Semantic Communications for NOMA Aided XR Applications
Chen, J., Wang, J., Jiang, C., & Wang, J. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3282836 · Cited by 38 · see citing works
Video and Audio Deepfake Datasets and Open Issues in Deepfake Technology: Being Ahead of the Curve
Akhtar, Z., Pendyala, T. L., & Athmakuri, V. S. (2024) · Forensic Sciences · doi:10.3390/forensicsci4030021 · Cited by 39 · see citing works
Generating Audio-Visual Slideshows from Text Articles Using Word Concreteness
Leake, M., Shin, H. V., Kim, J. O., & Agrawala, M. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376519 · Cited by 37 · see citing works
Self-citationAI-Powered MOOCs: Video Lecture Generation
Dao, X.-Q., Le, N.-B., & Nguyen, T.-M.-T. (2021) · International Conference on Image, Video and Signal Processing (IVSP) · doi:10.1145/3459212.3459227 · Cited by 35 · see citing works
Towards Realistic Visual Dubbing with Heterogeneous Sources
Xie, T., Liao, L., Bi, C., Tang, B., Yin, X., Yang, J., Wang, M., Yao, J., Zhang, Y., & Ma, Z. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475318 · Cited by 35 · see citing works
A comprehensive evaluation of feature-based AI techniques for deepfake detection
Sandotra, N., & Arora, B. (2024) · Neural Computing and Applications · doi:10.1007/s00521-023-09288-0 · Cited by 37 · see citing works
Multimodal Learning for Temporally Coherent Talking Face Generation With Articulator Synergy
Yu, L., Xie, H., & Zhang, Y. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3091863 · Cited by 34 · see citing works
PodReels: Human-AI Co-Creation of Video Podcast Teasers
Wang, S., Ning, Z., Truong, A., Dontcheva, M., Li, D., & Chilton, L. B. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661591 · Cited by 33 · see citing works
Text2video: Text-Driven Talking-Head Video Synthesis with Personalized Phoneme - Pose Dictionary
Zhang, S., Yuan, J., Liao, M., & Zhang, L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747380 · Cited by 34 · see citing works
Xia, H. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415845 · Cited by 32 · see citing works
Txt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text
Tandon, P., Chandak, S., Pataranutaporn, P., Liu, Y., Mapuranga, A. M., Maes, P., Weissman, T., & Sra, M. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221953 · Cited by 32 · see citing works
Iterative Text-Based Editing of Talking-Heads Using Neural Retargeting
Yao, X., Fried, O., Fatahalian, K., & Agrawala, M. (2021) · ACM Transactions on Graphics · doi:10.1145/3449063 · Cited by 31 · see citing works
Self-citationF³A-GAN: Facial Flow for Face Animation With Generative Adversarial Networks
Wu, X., Zhang, Q., Wu, Y., Wang, H., Li, S., Sun, L., & Li, X. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3112059 · Cited by 31 · see citing works
Automatic Instructional Video Creation from a Markdown-Formatted Tutorial
Chi, P., Frey, N., Panovich, K., & Essa, I. A. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474778 · Cited by 29 · see citing works
MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection
Coccomini, D. A., Zilos, G. K., Amato, G., Caldelli, R., Falchi, F., Papadopoulos, S., & Gennaro, C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3409054 · Cited by 33 · see citing works
Chen, W., Tan, X., Xia, Y., Qin, T., Wang, Y., & Liu, T.-Y. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413623 · Cited by 28 · see citing works
On the Generalization of Deep Learning Models in Video Deepfake Detection
Coccomini, D. A., Caldelli, R., Falchi, F., & Gennaro, C. (2023) · Journal of Imaging · doi:10.3390/jimaging9050089 · Cited by 32 · see citing works
“The fake images are manipulated using 15 different manipulations while the videos are manipulated using only 8 of them [27–36].”
MethodWatch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion
Agarwal, S., Hu, L., Ng, E., Darrell, T., Li, H., & Rohrbach, A. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00469 · Cited by 28 · see citing works
ExpressEdit: Video Editing with Natural Language and Sketching
Tilekbay, B., Yang, S., Lewkowicz, M. A., Suryapranata, A., & Kim, J. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645164 · Cited by 27 · see citing works
Sparse to Dense Motion Transfer for Face Image Animation
Zhao, R., Wu, T., & Guo, G. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00226 · Cited by 27 · see citing works
Deep learning applications on cybersecurity: A practical approach
Miranda-García, A., Rego, A. Z., Pastor-López, I., Sanz, B., Tellaeche, A., de la Puerta, J. G., & Bringas, P. G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126904 · Cited by 26 · see citing works
SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters
Ververas, E., & Zafeiriou, S. P. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01338-7 · Cited by 26 · see citing works
Automatic Video Creation From a Web Page
Chi, P., Sun, Z., Panovich, K., & Essa, I. A. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415814 · Cited by 25 · see citing works
Compositional Video Synthesis with Action GraphsPreprint
Bar, A., Herzig, R., Wang, X., Rohrbach, A., Chechik, G., Darrell, T., & Globerson, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15327 · Cited by 26 · see citing works
An Introduction to Digital Face Manipulation
Tolosana, R., Vera-Rodríguez, R., Fiérrez, J., Morales, A., & Ortega-García, J. (2022) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-030-87664-7_1 · Cited by 26 · see citing works
TellMeTalk: Multimodal-driven talking face video generation
Li, P., Zhao, H., Liu, Q., Tang, P., & Zhang, L. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2023.109049 · Cited by 22 · see citing works
Videoforensicshq: Detecting High-Quality Manipulated Face Videos
Fox, G., Liu, W., Kim, H., Seidel, H.-P., Elgharib, M., & Theobalt, C. (2021) · 2021 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428101 · Cited by 13 · see citing works
Self-citation
- Authors
- Fried, O., Tewari, A., Zollhöfer, M., Finkelstein, A., Shechtman, E., Goldman, D. B., Genova, K., Jin, Z., Theobalt, C., & Agrawala, M.
- Affiliations
- Stanford University (US)
- Max Planck Institute for Informatics (DE)
- Princeton University (US)
- Adobe Systems (United States) (US)
- ORCID
- Venue
- ACM Transactions on Graphics
- Published
- 2019-07-12
- Volume
- 38
- Issue
- 4
- Pages
- 1-14
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2948284812
- Cited by
- 270
- References
- 73
- Citations by year
- 2026: 14 · 2025: 30 · 2024: 36 · 2023: 51 · 2022: 46 · 2021: 47 · 2020: 39 · 2019: 7
- FWCI
- 15.55
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Graphics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Adobe Systems
- European Research Council (Consolidator Grant 4DRepLy #770784)
- Princeton University
- Brown Institute for Media Innovation
- National Science Foundation (NSF GRFP #DGE-1656466)
- Office of the Dean for Research, Princeton University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- speech modification, face geometry, reflectance estimation, facial expression estimation, video generation, speech translation, speech synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
