The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
On Face Segmentation, Face Swapping, and Face Perception
Abstract
We show that even when face images are unconstrained and arbitrarily paired, face swapping between them is quite simple. To this end, we make the following contributions. (a) Instead of tailoring systems for face segmentation, as others previously proposed, we show that a standard fully convolutional network (FCN) can achieve remarkably fast and accurate segmentations, provided that it is trained on a rich enough example set. For this purpose, we describe novel data collection and generation routines which provide challenging segmented face examples. (b) We use our segmentations for robust face swapping under unprecedented conditions. (c) Unlike previous work, our swapping is robust enough to allow for extensive quantitative tests. To this end, we use the Labeled Faces in the Wild (LFW) benchmark and measure the effect of intra- and inter-subject face swapping on recognition. We show that our intra-subject swapped faces remain as recognizable as their sources, testifying to the effectiveness of our method. In line with established perceptual studies, we show that better face swapping produces less recognizable inter-subject results. This is the first time this effect was quantitatively demonstrated by machine vision systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,822
EPnP: An Accurate O(n) Solution to the PnP Problem
Lepetit, V., Moreno-Noguer, F., & Fua, P. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0152-6 · Cited by 3,298
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831
Pérez, P., Gangnet, M., & Blake, A. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882269 · Cited by 2,020
Attribute and simile classifiers for face verification
Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459250 · Cited by 1,361
A 3D Face Model for Pose and Illumination Invariant Face Recognition
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. (2009) · 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance · doi:10.1109/avss.2009.58 · Cited by 1,326
Face Alignment Across Large Poses: A 3D Solution
Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.23 · Cited by 1,137
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.249 · Cited by 1,023
Robust Face Landmark Estimation under Occlusion
Burgos-Artizzu, X. P., Perona, P., & Dollar, P. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.191 · Cited by 775
Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A
Klare, B. F., Klein, B., Taborsky, E., Blanton, A., Cheney, J., Allen, K., Grother, P., Mah, A., Burge, M., & Jain, A. K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298803 · Cited by 701
Efficient hierarchical graph-based video segmentation
Grundmann, M., Kwatra, V., Han, M., & Essa, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539893 · Cited by 628
Effective face frontalization in unconstrained images
Hassner, T., Harel, S., Paz, E., & Enbar, R. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299058 · Cited by 613
Self-citationRegressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
Tran, A. T., Hassner, T., Masi, I., & Medioni, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.163 · Cited by 503
Self-citationFast Face-Swap Using Convolutional Neural Networks
Korshunova, I., Shi, W., Dambre, J., & Theis, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.397 · Cited by 501
Labeled Faces in the Wild: A Survey
Learned-Miller, E., Huang, G. B., RoyChowdhury, A., Li, H., & Hua, G. (2016) · Advances in Face Detection and Facial Image Analysis · doi:10.1007/978-3-319-25958-1_8 · Cited by 457
Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions
Bambach, S., Lee, S., Crandall, D. J., & Yu, C. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.226 · Cited by 408
Do We Really Need to Collect Millions of Faces for Effective Face Recognition?
Masi, I., Trần, A. T., Hassner, T., Leksut, J. T., & Medioni, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_35 · Cited by 339
Self-citationEmotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns
Levi, G., & Hassner, T. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2830587 · Cited by 336
Self-citationFace swapping: automatically replacing faces in photographs
Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., & Nayar, S. K. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360638 · Cited by 333
Surpassing Human-Level Face Verification Performance on LFW with GaussianFace
Lu, C., & Tang, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9797 · Cited by 303
Blanz, V., Scherbaum, K., Vetter, T., & Seidel, H. (2004) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2004.00799.x · Cited by 226
Hierarchical face parsing via deep learning
Luo, P., Wang, X., & Tang, X. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247963 · Cited by 181
Smith, B. M., Zhang, L., Brandt, J., Lin, Z., & Yang, J. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.447 · Cited by 175
Multi-objective convolutional learning for face labeling
Liu, S., Yang, J., Huang, C., & Yang, M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298967 · Cited by 157
Sinha, P., & Poggio, T. (1996) · Nature · doi:10.1038/384404a0 · Cited by 153
Viewing Real-World Faces in 3D
Hassner, T. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.448 · Cited by 151
Self-citationReal-Time Facial Segmentation and Performance Capture from RGB Input
Saito, S., Li, T., & Li, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_15 · Cited by 136
Kemelmacher-Shlizerman, I. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925871 · Cited by 79
PoseShop: Human Image Database Construction and Personalized Content Synthesis
Chen, T., Tan, P., Ma, L., Cheng, M., Shamir, A., & Hu, S. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.148 · Cited by 70
Robust Face Alignment Under Occlusion via Regional Predictive Power Estimation
Yang, H., He, X., Jia, X., & Patras, I. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2421438 · Cited by 60
Creating a Photoreal Digital Actor: The Digital Emily Project
Alexander, O., Mike Rogers, A., Lambeth, W., Chiang, M. J., & Debevec, P. E. (2009) · Conference on Visual Media Production · doi:10.1109/cvmp.2009.29 · Cited by 58
Photorealistic Face De-Identification by Aggregating Donors’ Face Components
Mosaddegh, S., Simon, L., & Jurie, F. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16811-1_11 · Cited by 53
Axelrod, V., & Yovel, G. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.027 · Cited by 51
Unsupervised face alignment by robust nonrigid mapping
Zhu, J., Gool, L. V., & Hoi, S. C. H. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459325 · Cited by 50
An eye for an eye: A single camera gaze-replacement method
Wolf, L., Freund, Z., & Avidan, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540133 · Cited by 42
Manifold Learning and Spectral Clustering for Image Phylogeny Forests
Oikawa, M. A., Dias, Z., Rocha, A. D. R., & Goldenstein, S. K. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2442527 · Cited by 40
Creating a Photoreal Digital Actor: The Digital Emily Project
Alexander, O., Mike Rogers, A., Lambeth, W., Chiang, M. J., & Debevec, P. E. (2009) · Conference on Visual Media Production · doi:10.1109/cvmp.2009.23 · Cited by 39
Using Segmentation to Predict the Absence of Occluded Parts
Ghiasi, G., & Fowlkes, C. C. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.22 · Cited by 27
Face replacement with large-pose differences
Lin, Y., Lin, Q., Tang, F., & Wang, S. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2396426 · Cited by 15
Structured Semi-supervised Forest for Facial Landmarks Localization with Face Mask Reasoning
Jia, X., Yang, H., Chan, K.-P., & Patras, I. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.85 · Cited by 14
Cited by
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706
Self-citation
Metadata
- Authors
- Nirkin, Y., Masi, I., Tran Tuan, A., Hassner, T., & Medioni, G.
- Venue
- 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 98-105
- ISSN
- 1541-5058 (ISSN-L)
- OpenAlex
- W2608058963
- Cited by
- 285
- References
- 40
- FWCI
- 10.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.