The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision
Abstract
The estimation of 3D face shape from a single image must be robust to variations in lighting, head pose, expression, facial hair, makeup, and occlusions. Robustness requires a large training set of in-the-wild images, which by construction, lack ground truth 3D shape. To train a network without any 2D-to-3D supervision, we present RingNet, which learns to compute 3D face shape from a single image. Our key observation is that an individual’s face shape is constant across images, regardless of expression, pose, lighting, etc. RingNet leverages multiple images of a person and automatically detected 2D face features. It uses a novel loss that encourages the face shape to be similar when the identity is the same and different for different people. We achieve invariance to expression by representing the face using the FLAME model. Once trained, our method takes a single image and outputs the parameters of FLAME, which can be readily animated. Additionally we create a new database of faces “not quite in-the-wild” (NoW) with 3D head scans and high-resolution images of the subjects in a wide variety of conditions. We evaluate publicly available methods and find that RingNet is more accurate than methods that use 3D supervision. The dataset, model, and results are available for research purposes at http://ringnet.is.tuebingen.mpg.de.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
VGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Self-citationExpressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationFace2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., Blitzer, J. C., & Saul, L. K. (2005) · Cited by 1,690
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277
Learning a model of facial shape and expression from 4D scans
Li, T., Bolkart, T., Black, M. J., Li, H., & Romero, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130813 · Cited by 1,223
Self-citationFace Alignment Across Large Poses: A 3D Solution
Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.23 · Cited by 1,137
Hand Keypoint Detection in Single Images Using Multiview Bootstrapping
Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.494 · Cited by 1,119
Human Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_33 · Cited by 767
S^3FD: Single Shot Scale-Invariant Face Detector
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., & Li, S. Z. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.30 · Cited by 653
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
Generating 3D Faces Using Convolutional Mesh Autoencoders
Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_43 · Cited by 560
Self-citationRegressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
Tran, A. T., Hassner, T., Masi, I., & Medioni, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.163 · Cited by 503
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
Jackson, A. S., Bulat, A., Argyriou, V., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.117 · Cited by 449
Nonlinear 3D Face Morphable Model
Tran, L., & Liu, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00767 · Cited by 407
Displaced dynamic expression regression for real-time facial tracking and animation
Cao, C., Hou, Q., & Zhou, K. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601204 · Cited by 356
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.401 · Cited by 343
Unsupervised Training for 3D Morphable Model Regression
Genova, K., Cole, F., Maschinot, A., Sarna, A., Vlasic, D., & Freeman, W. T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00874 · Cited by 311
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications
Zollhöfer, M., Thies, J., Garrido, P., Bradley, D., Beeler, T., Pérez, P., Stamminger, M., Nießner, M., & Theobalt, C. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13382 · Cited by 307
SfSNet: Learning Shape, Reflectance and Illuminance of Faces 'in the Wild'
Sengupta, S., Kanazawa, A., Castillo, C. D., & Jacobs, D. W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00659 · Cited by 287
Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation
Sela, M., Richardson, E., & Kimmel, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.175 · Cited by 280
Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz
Tewari, A., Zollhofer, M., Garrido, P., Bernard, F., Kim, H., Perez, P., & Theobalt, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00270 · Cited by 270
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.153 · Cited by 269
Morphable Face Models - An Open Framework
Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M., Schoenborn, S., & Vetter, T. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00021 · Cited by 268
Reconstruction of Personalized 3D Face Rigs from Monocular Video
Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2890493 · Cited by 260
Extreme 3D Face Reconstruction: Seeing Through Occlusions
Tran, A. T., Hassner, T., Masi, I., Paz, E., Nirkin, Y., & Medioni, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00414 · Cited by 183
Face reconstruction in the wild
Kemelmacher-Shlizerman, I., & Seitz, S. M. (2011) · International Conference on Computer Vision · doi:10.1109/iccv.2011.6126439 · Cited by 133
A 3D Morphable Model of Craniofacial Shape and Texture Variation
Dai, H., Pears, N. E., Smith, W. A. P., & Duncan, C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.335 · Cited by 120
InverseFaceNet: Deep Monocular Inverse Face Rendering
Kim, H., Zollhöfer, M., Tewari, A., Thies, J., Richardt, C., & Theobalt, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00486 · Cited by 100
Total Moving Face Reconstruction
Suwajanakorn, S., Kemelmacher-Shlizerman, I., & Seitz, S. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_52 · Cited by 93
Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondences
Bas, A., Smith, W. A. P., Bolkart, T., & Wuhrer, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54427-4_28 · Cited by 88
Self-citationEvaluation of Dense 3D Reconstruction from 2D Face Images in the Wild
Feng, Z., Huber, P., Kittler, J., Hancock, P., Wu, X., Zhao, Q., Koppen, P., & Rätsch, M. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00123 · Cited by 61
Cited by
3D Morphable Face Models—Past, Present, and Future
Egger, B., Smith, W. A. P., Tewari, A., Wuhrer, S., Zollhoefer, M., Beeler, T., Bernard, F., Bolkart, T., Kortylewski, A., Romdhani, S., et al. (2020) · ACM Transactions on Graphics · doi:10.1145/3395208 · Cited by 448
Self-citationHifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping
Wang, Y., Chen, X., Zhu, J., Chu, W., Tai, Y., Wang, C., Li, J., Wu, Y., Huang, F., & Ji, R. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/157 · Cited by 138
Metadata
- Authors
- Sanyal, S., Bolkart, T., Feng, H., & Black, M. J.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 7755-7764
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2945729334
- Cited by
- 332
- References
- 43
- FWCI
- 11.55
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.