CoChat
Sign Up
Cited by 418Generative Adversarial Networks and Image SynthesisFace recognition and analysisDigital Media Forensic Detection

SimSwap: An Efficient Framework For High Fidelity Face Swapping

Open access
Chen, Renwang, Chen, Xuanhong, Ni, Bingbing, Ge, Yanhao·2020·Proceedings of the 28th ACM International Conference on Multimedia
DOI 10.1145/3394171.3413630
Open this paper in CoChat — free accountOpen PDF

Abstract

We propose an efficient framework, called Simple Swap (SimSwap), aiming for generalized and high fidelity face swapping. In contrast to previous approaches that either lack the ability to generalize to arbitrary identity or fail to preserve attributes like facial expression and gaze direction, our framework is capable of transferring the identity of an arbitrary source face into an arbitrary target face while preserving the attributes of the target face. We overcome the above defects in the following two ways. First, we present the ID Injection Module (IIM) which transfers the identity information of the source face into the target face at feature level. By using this module, we extend the architecture of an identity-specific face swapping algorithm to a framework for arbitrary face swapping. Second, we propose the Weak Feature Matching Loss which efficiently helps our framework to preserve the facial attributes in an implicit way. Extensive experiments on wild faces demonstrate that our SimSwap is able to achieve competitive identity performance while preserving attributes better than previous state-of-the-art methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3394171.3413630

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Target-oriented methods (FaceShifter and SimSwap) transfer identity information of the source face to the driving face at the feature level and are more difficult to detect than source-oriented FSGAN that processes the driving face at the image level and has blending artifacts.

    “the target-oriented methods (FaceShifter and SimSwap) [8], which transfer the identity information of the source face to the driving face at the feature level, are more diicult to detect than the source-oriented FSGAN that processes the driving face at the image level and has blending artifacts.”

    From DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models (2024) · cites this paper as [8]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 25

  1. [1][1] Martín Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. CoRR abs/1701.07875 (2017). arXiv:1701.07875 http://arxiv.org/abs/1701.07875 - · arXiv
  2. [2][2] Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. 2018. Towards Open-Set Identity Preserving Face Synthesis. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 . IEEE Computer Society, 6713-6722. https://doi.org/10.1109/CVPR.2018.00702 - · DOI
  3. [4][4] Volker Blanz, Kristina Scherbaum, Thomas Vetter, and Hans-Peter Seidel. 2004. Exchanging Faces in Images. Comput. Graph. Forum 23, 3 (2004), 669-676. https: //doi.org/10.1111/j.1467-8659.2004.00799.x - · DOI
  4. [5][5] Andrew Brock, Jeff Donahue, and Karen Simonyan. 2019. Large Scale GAN Training for High Fidelity Natural Image Synthesis. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net. https://openreview.net/forum?id=B1xsqj09Fm -
  5. [10][10] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David WardeFarley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada , Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (Eds.). 2672-2680. http://papers. nips.cc/paper/5423-generative-adversarial-nets -
  6. [11][11] Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville. 2017. Improved Training of Wasserstein GANs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA , Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5767-5777. http://papers.nips.cc/paper/7159-improved-training-of-wasserstein-gans -
  7. [13][13] Xun Huang and Serge J. Belongie. 2017. Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017 . IEEE Computer Society, 1510-1519. https://doi.org/10.1109/ICCV.2017.167 - · DOI · Paper page
  8. [15][15] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-toImage Translation with Conditional Adversarial Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 . IEEE Computer Society, 5967-5976. https://doi.org/10.1109/ CVPR.2017.632 -
  9. [14][14] Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 (JMLR Workshop and Conference Proceedings) , Francis R. Bach and David M. Blei (Eds.), Vol. 37. JMLR.org, 448-456. http://proceedings.mlr.press/ v37/ioffe15.html -
  10. [18][18] Iryna Korshunova, Wenzhe Shi, Joni Dambre, and Lucas Theis. 2017. Fast FaceSwap Using Convolutional Neural Networks. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017 . IEEE Computer Society, 3697-3705. https://doi.org/10.1109/ICCV.2017.397 - · DOI · Paper page
  11. [17][17] Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6980 - · arXiv
  12. [19][19] Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. 2019. MaskGAN: Towards Diverse and Interactive Facial Image Manipulation. CoRR abs/1907.11922 (2019). 20. [arXiv:1907.11922 http://arxiv.org/abs/1907.11922](http://arxiv.org/abs/1907.11922) - · arXiv
  13. [21][21] Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. 2019. Few-Shot Unsupervised Image-to-Image Translation. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019 . IEEE, 10550-10559. https://doi.org/ 10.1109/ICCV.2019.01065 - · DOI
  14. [20][20] Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. 2019. FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping. CoRR abs/1912.13457 (2019). arXiv:1912.13457 http://arxiv.org/abs/1912.13457 - · arXiv
  15. [22][22] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep Learning Face Attributes in the Wild. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015 . IEEE Computer Society, 3730-3738. https://doi.org/10.1109/ICCV.2015.425 - · DOI · Paper page
  16. [24][24] Ryota Natsume, Tatsuya Yatagawa, and Shigeo Morishima. 2018. FSNet: An Identity-Aware Generative Model for Image-Based Face Swapping. In Computer Vision - ACCV 2018 - 14th Asian Conference on Computer Vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part VI (Lecture Notes in Computer Science) , C. V. Jawahar, Hongdong Li, Greg Mori, and Konrad Schindler (Eds.), Vol. 11366. Springer, 117-132. https://doi.org/10.1007/978-3-030-20876-9\_8 - · DOI
  17. [25][25] Ryota Natsume, Tatsuya Yatagawa, and Shigeo Morishima. 2018. RSGAN: face swapping and editing using face and hair representation in latent spaces. In Special Interest Group on Computer Graphics and Interactive Techniques Conference, SIGGRAPH 2018, Vancouver, BC, Canada, August 12-16, 2018, Posters Proceedings . ACM, 69:1-69:2. https://doi.org/10.1145/3230744.3230818 - · DOI · Paper page
  18. [27][27] Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gérard G. Medioni. 2018. On Face Segmentation, Face Swapping, and Face Perception. In 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, Xi'an, China, May 15-19, 2018 . IEEE Computer Society, 98-105. https://doi.org/10.1109/ FG.2018.00024 -
  19. [26][26] Yuval Nirkin, Yosi Keller, and Tal Hassner. 2019. FSGAN: Subject Agnostic Face Swapping and Reenactment. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019 . IEEE, 7183-7192. https://doi.org/10.1109/ICCV.2019.00728 - · DOI · Paper page
  20. [28][28] Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. 2019. Semantic Image Synthesis With Spatially-Adaptive Normalization. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 . Computer Vision Foundation / IEEE, 2337-2346. https://doi.org/10. 1109/CVPR.2019.00244 -
  21. [30][30] Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson image editing. ACM Trans. Graph. 22, 3 (2003), 313-318. https://doi.org/10.1145/882262.882269 - · DOI
  22. [29][29] Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. 2009. A 3D Face Model for Pose and Illumination Invariant Face Recognition. In Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, 2-4 September 2009, Genova, Italy , Stefano Tubaro and Jean-Luc Dugelay (Eds.). IEEE Computer Society, 296-301. https://doi.org/10.1109/AVSS.2009.58 - · DOI
  23. [31][31] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE International Conference on Computer Vision . 1-11. -
  24. [32][32] Nataniel Ruiz, Eunji Chong, and James M Rehg. 2018. Fine-grained head pose estimation without keypoints. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops . 2074-2083. -
  25. [34][34] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. High-Resolution Image Synthesis and Semantic Manipulation With Conditional GANs. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018 . IEEE Computer Society, 8798-8807. https://doi.org/10.1109/CVPR.2018.00917 · DOI

Cited by

Sorted by citations

Authors
Chen, R., Chen, X., Ni, B., & Ge, Y.
Affiliations
  • Shanghai Jiao Tong University (CN)
  • Tencent (China) (CN)
Venue
Proceedings of the 28th ACM International Conference on Multimedia
Publisher
ACM
Published
2020-10-12
Pages
2003-2011
DOI
10.1145/3394171.3413630
OpenAlex
W3092709185
Cited by
418
References
6
Citations by year
2026: 75 · 2025: 123 · 2024: 105 · 2023: 66 · 2022: 40 · 2021: 8 · 2020: 1
FWCI
8.87
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM International Conference on Multimedia (ACM MM) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
face swapping, identity transfer, facial expressions, gaze direction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review