The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Face2Face: real-time face capture and reenactment of RGB videos
Abstract
Face2Face is an approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time. This live setup has also been shown at SIGGRAPH Emerging Technologies 2016, by Thies et al. where it won the Best in Show Award.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deformable Model Fitting by Regularized Landmark Mean-Shift
Saragih, J. M., Lucey, S., & Cohn, J. F. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0380-4 · Cited by 787
Video Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669
Realtime performance-based facial animation
Weise, T., Bouaziz, S., Li, H., & Pauly, M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964972 · Cited by 356
Cited by
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
Self-citationFake News Detection on Social Media: A Data Mining Perspective
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3137597.3137600 · Cited by 3,304
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846
“Although in recent years there have been many sophisticated algorithms for generating realistic synthetic face videos [9, 13, 46, 51, 26, 47, 37, 20, 23, 10, 21, 50], most of these have not been in mainstream as open-source software tools that anyone can use.”
BackgroundExposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
Self-citationFake News Detection on Social Media: A Data Mining Perspective
Shu, K., Sliva, A. L., Wang, S., Tang, J., & Liu, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01967 · Cited by 581
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00500 · Cited by 537
“It contains 1.8 million manipulated frames and 4,000 fake videos generated using two face swapping algorithms, DeepFakes [1] and FaceSwap [2], and two face reenactment methods, Face2Face [52] and NeuralTextures [51].”
BackgroundMethodInfluentialDetecting Deepfakes with Self-Blended Images
Shiohara, K., & Yamasaki, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01816 · Cited by 503
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
Gafni, G., Thies, J., Zollhofer, M., & Niesner, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00854 · Cited by 501
Self-citationFakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443
State of the Art on Neural Rendering
Tewari, A., Fried, O., Thies, J., Sitzmann, V., Lombardi, S., Sunkavalli, K., Martin‐Brualla, R., Simon, T., Saragih, J., Nießner, M., et al. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14022 · Cited by 430
Self-citationDeepFake Detection by Analyzing Convolutional Traces
Guarnera, L., Giudice, O., & Battiato, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00341 · Cited by 374
“[36] proposed an automated benchmark for fake detection, based mainly on four manipulation methods: two computer graphics-based methods (Face2Face [40], FaceSwap 7) and 2 learning-based approaches (DeepFakes 8, NeuralTextures [39]).”
MethodM2TR: Multi-modal Multi-scale Transformers for Deepfake Detection
Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., & Li, S.-N. (2022) · Proceedings of the 2022 International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531415 · Cited by 368
Learning Self-Consistency for Deepfake Detection
Zhao, T., Xu, X., Xu, M., Ding, H., Xiong, Y., & Xia, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01475 · Cited by 383
Animating Arbitrary Objects via Deep Motion Transfer
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00248 · Cited by 349
Local Relation Learning for Face Forgery Detection
Chen, S., Yao, T., Chen, Y., Ding, S., Li, J., & Ji, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16193 · Cited by 334
Recycle-GAN: Unsupervised Video Retargeting
Bansal, A., Ma, S., Ramanan, D., & Sheikh, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_8 · Cited by 328
Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection
Chen, L., Zhang, Y., Song, Y., Liu, L., & Wang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01815 · Cited by 307
“The realistic image generation brought by the generative adversarial network (GAN) raises a security issue that human portraits can be easily substituted to provide malicious bioinformatics [11, 45, 15, 44, 51, 56, 40, 54, 48].”
BackgroundMethodThe Epistemic Threat of Deepfakes
Fallis, D. (2021) · Philosophy & Technology · doi:10.1007/s13347-020-00419-2 · Cited by 266
“For example, many people have thought that the videos of the Apollo moon landings were faked (see Villard 2004).”
BackgroundFaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2019) · arXiv · doi:10.48550/arxiv.1912.13457 · Cited by 236
ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., & Verdoliva, L. (2018) · arXiv · doi:10.48550/arxiv.1812.02510 · Cited by 221
Self-citationJoint Audio-Visual Deepfake Detection
Zhou, Y., & Lim, S.-N. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01453 · Cited by 217
“FF [39]: Faceforensics++ is a deepfake benchmark dataset comprising 5000 video sequences with audio channels, where the video channel is manipulated with 4 methods: 2youtu.be/SK4vGKxm1PY Deepfakes [1], Face2Face [47], FaceSwap [2] and NeuralTexture [46].”
ID-Reveal: Identity-aware DeepFake Video Detection
Cozzolino, D., Rossler, A., Thies, J., Niesner, M., & Verdoliva, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01483 · Cited by 221
Self-citationCountering Malicious DeepFakes: Survey, Battleground, and Horizon
Juefei-Xu, F., Wang, R., Huang, Y., Guo, Q., Ma, L., & Liu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01606-8 · Cited by 191
TweepFake: about detecting deepfake tweets
Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251415 · Cited by 185
“Deepfake technologies have first risen in the computer vision field [20–23], followed by effective attempts on audio manipulation [24,25] and text generation [2].”
BackgroundMTD-Net: Learning to Detect Deepfakes Images by Multi-Scale Texture Difference
Yang, J., Li, A., Xiao, S., Lu, W., & Gao, X. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3102487 · Cited by 184
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Haliassos, A., Mira, R., Petridis, S., & Pantic, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01453 · Cited by 180
Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection
Yan, Z., Luo, Y., Lyu, S., Liu, Q., & Wu, B. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00858 · Cited by 166
DeepFake video detection using convolutional vision transformer
Wodajo, D., & Atnafu, S. (2021) · arXiv · doi:10.48550/arxiv.2102.11126 · Cited by 147
FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning
Kim, M., Tariq, S., & Woo, S. S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00111 · Cited by 118
TALL: Thumbnail Layout for Deepfake Video Detection
Xu, Y., Liang, J., Jia, G., Yang, Z., Zhang, Y., & He, R. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02071 · Cited by 109
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
Trinh, L., Tsang, M., Rambhatla, S., & Liu, Y. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00202 · Cited by 115
Lahiri, A., Kwatra, V., Frueh, C., Lewis, J., & Bregler, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00278 · Cited by 101
“Audio-driven Video Synthesis: These methods aim to generate visually plausible 2D talking head videos, conditioned on novel audio.”
BackgroundMethodDeepfakeBench: A Comprehensive Benchmark of Deepfake Detection
Yan, Z., Zhang, Y., Yuan, X., Lyu, S., & Wu, B. (2023) · arXiv · doi:10.48550/arxiv.2307.01426 · Cited by 97
“NeuralTextures [41] and Face2Face [42], utilized in FaceForensics++, stand out as standard face-reenactment methods.”
MethodOne Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework
Tariq, S., Lee, S., & Woo, S. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449809 · Cited by 91
“Our work’s primary focus is to investigate a training strategy and classifier that can detect deepfakes from multiple benchmark datasets (e.g., FaceSwap, DeepFake, and Face2Face) and unknown generation methods (DFW videos).”
BackgroundMethodInfluentialDetecting Deepfakes with Metric Learning
Kumar, A., Bhavsar, A., & Verma, R. (2020) · 2020 8th International Workshop on Biometrics and Forensics (IWBF) · doi:10.1109/iwbf49977.2020.9107962 · Cited by 90
Deepfake Video Detection with Spatiotemporal Dropout Transformer
Zhang, D., Lin, F., Hua, Y., Wang, P., Zeng, D., & Ge, S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3547913 · Cited by 50
Detection and Localization of Facial Expression Manipulations
Mazaheri, G., & Roy-Chowdhury, A. K. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00283 · Cited by 44
Modality Dropout for Improved Performance-driven Talking Faces
Hussen Abdelaziz, A., Theobald, B.-J., Dixon, P., Knothe, R., Apostoloff, N., & Kajareker, S. (2020) · Proceedings of the 2020 International Conference on Multimodal Interaction · doi:10.1145/3382507.3418840 · Cited by 42
Generalized zero and few-shot transfer for facial forgery detection
Aneja, S., & Nießner, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11863 · Cited by 31
Self-citationEnhancing General Face Forgery Detection via Vision Transformer with Low-Rank Adaptation
Kong, C., Li, H., & Wang, S. (2023) · 2023 IEEE 6th International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr59079.2023.00033 · Cited by 33
Quality-based Artifact Modeling for Facial Deepfake Detection in Videos
Concas, S., La Cava, S. M., Casula, R., Orrù, G., Puglisi, G., & Marcialis, G. L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00389 · Cited by 12
“The last two reenactment methods are a learning-based and a graphics-based approach named NeuralTex-tures [29] and Face2Face [30], respectively.”
MethodKrasilnikov, M., Nikitin, M., & Konushin, A. (2025) · The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprs-archives-xlviii-2-w9-2025-169-2025 · Cited by 2
“Face2Face (Thies et al., 2016) pioneered real-time facial reenactment by mapping expressions from a source to a target face with high fidelity.”
Han, Y.-H., Huang, T.-M., Hua, K.-L., & Chen, J.-C. (2024) · arXiv · doi:10.48550/arxiv.2404.05583 · Cited by 45
Metadata
- Authors
- Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M.
- Venue
- Communications of the ACM
- Published
- 2018-12-19
- Volume
- 62
- Issue
- 1
- Pages
- 96-104
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2906447902
- Cited by
- 244
- References
- 3
- FWCI
- 14.70
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.