Effective face frontalization in unconstrained images
Abstract
“Frontalization” is the process of synthesizing frontal facing views of faces appearing in single unconstrained photos. Recent reports have suggested that this process may substantially boost the performance of face recognition systems. This, by transforming the challenging problem of recognizing faces viewed from unconstrained viewpoints to the easier problem of recognizing faces in constrained, forward facing poses. Previous frontalization methods did this by attempting to approximate 3D facial shapes for each query image. We observe that 3D face shape estimation from unconstrained photos may be a harder problem than frontalization and can potentially introduce facial misalignments. Instead, we explore the simpler approach of using a single, unmodified, 3D surface as an approximation to the shape of all input faces. We show that this leads to a straightforward, efficient and easy to implement method for frontalization. More importantly, it produces aesthetic new frontal views and is surprisingly effective when used for face recognition and gender estimation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
More advanced face frontalization involving approximation of 3D shape is useful when dealing with 3D head pose variation.
“More advanced face frontalization involving the approximation of 3D shape is useful when dealing with 3D head pose variation [104], [105].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [104]
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · see citing works
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565 · see citing works
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070 · see citing works
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398 · see citing works
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097 · see citing works
Wolpert, D. H. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80023-1 · Cited by 7,951 · see citing works
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · see citing works
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. R. (1999) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/311535.311556 · Cited by 4,695 · see citing works
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445 · see citing works
Deep Learning Face Representation from Predicting 10,000 Classes
Sun, Y. J., Wang, X., & Tang, X. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.244 · Cited by 1,995 · see citing works
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & la Torre, F. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.75 · Cited by 1,917 · see citing works
Deep Learning Face Representation by Joint Identification-Verification
Sun, Y., Wang, X., & Tang, X. (2014) · arXiv · doi:10.48550/arxiv.1406.4773 · Cited by 1,906 · see citing works
A Theory of Shape by Space Carving
Kutulakos, K. N., & Seitz, S. M. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008191222954 · Cited by 1,281 · see citing works
Age and Gender Estimation of Unfiltered Faces
Eidinger, E., Enbar, R., & Hassner, T. (2014) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2014.2359646 · Cited by 814 · see citing works
Self-citationDiscriminative Deep Metric Learning for Face Verification in the Wild
Hu, J., Lu, J., & Tan, Y. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.242 · Cited by 697 · see citing works
Anthropometry of the head and face in medicine
(1982) · Journal of Oral and Maxillofacial Surgery · doi:10.1016/0278-2391(82)90269-5 · Cited by 472 · see citing works
Descriptor Based Methods in the Wild
Wolf, L., Hassner, T., & Taigman, Y. (2008) · HAL · Cited by 447 · see citing works
Self-citationFisher Vector Faces in the Wild
Simonyan, K., Parkhi, O., Vedaldi, A., & Zisserman, A. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.8 · Cited by 446 · see citing works
3D Face Reconstruction from a Single Image Using a Single Reference Face Shape
Kemelmacher-Shlizerman, I., & Basri, R. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.63 · Cited by 440 · see citing works
FRVT 2006 and ICE 2006 large-scale results
Phillips, P. J., Scruggs, W., O’Toole, A. J., et al. (2007) · doi:10.6028/nist.ir.7408 · Cited by 418 · see citing works
Displaced dynamic expression regression for real-time facial tracking and animation
Cao, C., Hou, Q., & Zhou, K. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601204 · Cited by 378 · see citing works
Unsupervised Joint Alignment of Complex Images
Huang, G. B., Jain, V., & Miller, E. G. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408858 · Cited by 340 · see citing works
Wolf, L., Hassner, T., & Taigman, Y. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.230 · Cited by 332 · see citing works
Self-citationOjala, T., Pietikäinen, M., & Mäenpää, T. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44732-6_41 · Cited by 322 · see citing works
Surpassing Human-Level Face Verification Performance on LFW with GaussianFace
Lu, C., & Tang, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9797 · Cited by 307 · see citing works
Hybrid Deep Learning for Face Verification
Sun, Y. J., Wang, X., & Tang, X. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.188 · Cited by 292 · see citing works
Similarity Scores Based on Background Samples
Wolf, L., Hassner, T., & Taigman, Y. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12304-7_9 · Cited by 282 · see citing works
Self-citationLearning to Align from Scratch
Huang, G. B., Mattar, M., Lee, H., & Miller, E. G. (2012) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 270 · see citing works
Similarity Metric Learning for Face Recognition
Cao, Q., Ying, Y., & Li, P. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.299 · Cited by 237 · see citing works
Blanz, V., Scherbaum, K., Vetter, T. R., & Seidel, H. (2004) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2004.00799.x · Cited by 231 · see citing works
Viewing Real-World Faces in 3D
Hassner, T. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.448 · Cited by 150 · see citing works
Self-citationExpression flow for 3D-aware face component transfer
Yang, F. H., Wang, J., Shechtman, E., Bourdev, L. D., & Metaxas, D. (2011) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1964921.1964955 · Cited by 128 · see citing works
The One-Shot similarity kernel
Wolf, L., Hassner, T., & Taigman, Y. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459323 · Cited by 108 · see citing works
Self-citationExpression flow for 3D-aware face component transfer
Yang, F. H., Wang, J., Shechtman, E., Bourdev, L. D., & Metaxas, D. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964955 · Cited by 106 · see citing works
Statistical Symmetric Shape from Shading for 3D Structure Recovery of Faces
Dovgard, R., & Basri, R. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24671-8_8 · Cited by 105 · see citing works
Recover Canonical-View Faces in the Wild with Deep Neural NetworksPreprint
Zhu, Z., Luo, P., Wang, X., & Tang, X. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.3543 · Cited by 102 · see citing works
Real-time avatar animation from a single image
Saragih, J., Lucey, S., & Cohn, J. F. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771383 · Cited by 76 · see citing works
Real-time conversion from a single 2D face image to a 3D text-driven emotive audio-visual avatar
Tang, H., Hu, Y., Fu, Y., et al. (2008) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2008.4607657 · Cited by 29 · see citing works
Progressive surface reconstruction from images using a local prior
Zeng, G., Paris, S., Quan, L., & Sillion, F. X. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.196 · Cited by 26 · see citing works
When standard RANSAC is not enough: cross-media visual matching with hypothesis relevancy
Hassner, T., Assif, L., & Wolf, L. (2013) · Machine Vision and Applications · doi:10.1007/s00138-013-0571-4 · Cited by 23 · see citing works
Self-citationSymmetry-Aided Frontal View Synthesis for Pose-Robust Face Recognition
González-Jiménez, D., & Alba‐Castro, J. L. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366216 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706 · see citing works
Self-citationDisentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
Deng, Y., Yang, J., Chen, D., Wen, F., & Tong, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00520 · Cited by 329 · see citing works
Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns
Levi, G., & Hassner, T. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2830587 · Cited by 336 · see citing works
Self-citationDeep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
A survey on face data augmentation for the training of deep neural networks
Wang, X., Wang, K., & Lian, S. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04748-3 · Cited by 141 · see citing works
HoloNet: towards robust emotion recognition in the wild
Yao, A., Cai, D., Hu, P., Wang, S., Sha, L., & Chen, Y. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997639 · Cited by 115 · see citing works
Juefei-Xu, F., Luu, K., & Savvides, M. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2468173 · Cited by 74 · see citing works
Audio and face video emotion recognition in the wild using deep neural networks and small datasets
Ding, W., Xu, M., Huang, D., Lin, W., Dong, M., Yu, X., & Li, H. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997637 · Cited by 51 · see citing works
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Peng, K., Mathur, A., & Narayanan, A. (2021) · arXiv · doi:10.48550/arxiv.2108.02922 · Cited by 31 · see citing works
- Authors
- Hassner, T., Harel, S., Paz, E., & Enbar, R.
- Affiliations
- Open University of Israel (IL)
- ORCID
- Hassner, Tal
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 4295-4304
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1916406603
- Cited by
- 613
- References
- 41
- Citations by year
- 2026: 3 · 2025: 18 · 2024: 26 · 2023: 40 · 2022: 54 · 2021: 69 · 2020: 74 · 2019: 84 · 2018: 83 · 2017: 85 · 2016: 56 · 2015: 21
- FWCI
- 33.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- Office of the Director of National Intelligence
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- face frontalization, unconstrained face images, face recognition, gender prediction, frontal view
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review