The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Open accessAbstract
In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image. To this end, we combine a convolutional encoder network with an expert-designed generative model that serves as decoder. The core innovation is the differentiable parametric decoder that encapsulates image formation analytically based on a generative model. Our decoder takes as input a code vector with exactly defined semantic meaning that encodes detailed face pose, shape, expression, skin reflectance and scene illumination. Due to this new way of combining CNN-based with model-based face reconstruction, the CNN-based encoder learns to extract semantically meaningful parameters from a single monocular input image. For the first time, a CNN encoder and an expert-designed generative model can be trained end-to-end in an unsupervised manner, which renders training on very large (unlabeled) real world data feasible. The obtained reconstructions compare favorably to current state-of-the-art approaches in terms of quality and richness of representation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
Bradski, G. R. (2000) · Medical Entomology and Zoology · Cited by 5,994
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction
Masci, J., Meier, U., Cireşan, D., & Schmidhuber, J. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21735-7_7 · Cited by 1,955
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Self-citationDeep Convolutional Network Cascade for Facial Point Detection
Sun, Y., Wang, X., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.446 · Cited by 1,382
Deformation transfer for triangle meshes
Sumner, R. W., & Popović, J. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015736 · Cited by 1,052
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.249 · Cited by 1,023
Deformable Model Fitting by Regularized Landmark Mean-Shift
Saragih, J. M., Lucey, S., & Cohn, J. F. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0380-4 · Cited by 787
Deep Convolutional Inverse Graphics Network
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03167 · Cited by 653
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
Self-citationRegressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
Tran, A. T., Hassner, T., Masi, I., & Medioni, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.163 · Cited by 503
Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment
Zhang, J., Shan, S., Kan, M., & Chen, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_1 · Cited by 468
Robust Face Recognition via Multimodal Deep Face Representation
Ding, C., & Tao, D. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477042 · Cited by 449
An All-In-One Convolutional Neural Network for Face Analysis
Ranjan, R., Sankaranarayanan, S., Castillo, C. D., & Chellappa, R. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.137 · Cited by 409
Reanimating Faces in Images and Video
Blanz, V., Basso, C., Poggio, T., & Vetter, T. (2003) · Computer Graphics Forum · doi:10.1111/1467-8659.t01-1-00712 · Cited by 387
Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade
Zhou, E., Fan, H., Cao, Z., Jiang, Y., & Yin, Q. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.58 · Cited by 356
Learning Detailed Face Reconstruction from a Single Image
Richardson, E., Sela, M., Or-El, R., & Kimmel, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.589 · Cited by 336
Deep Learning Identity-Preserving Face Space
Zhu, Z., Luo, P., Wang, X., & Tang, X. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.21 · Cited by 312
3D Face Reconstruction by Learning from Synthetic Data
Richardson, E., Sela, M., & Kimmel, R. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.56 · Cited by 309
Romdhani, S., & Vetter, T. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.145 · Cited by 299
The First Facial Landmark Tracking in-the-Wild Challenge: Benchmark and Results
Shen, J., Zafeiriou, S., Chrysos, G. G., Kossaifi, J., Tzimiropoulos, G., & Pantic, M. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.132 · Cited by 292
Reconstruction of Personalized 3D Face Rigs from Monocular Video
Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2890493 · Cited by 260
Self-citationStacked Progressive Auto-Encoders (SPAE) for Face Recognition Across Poses
Kan, M., Shan, S., Chang, H., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.243 · Cited by 243
Facial feature point detection: A comprehensive survey
Wang, N., Gao, X., Tao, D., Yang, H., & Li, X. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.05.013 · Cited by 238
Deep Cascaded Bi-Network for Face Hallucination
Zhu, S., Liu, S., Loy, C. C., & Tang, X. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_37 · Cited by 237
Project-Out Cascaded Regression with an application to face alignment
Tzimiropoulos, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298989 · Cited by 218
Single Sample Face Recognition via Learning Deep Supervised Autoencoders
Gao, S., Zhang, Y., Jia, K., Lu, J., & Zhang, Y. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2446438 · Cited by 214
DenseReg: Fully Convolutional Dense Shape Regression In-the-WildPreprint
G\"uler, R. A., Trigeorgis, G., Antonakos, E., Snape, P., Zafeiriou, S. P., & Kokkinos, I. (2016) · arXiv (Cornell University) · Cited by 192
Robust LSTM-Autoencoders for Face De-Occlusion in the Wild
Zhao, F., Feng, J., Zhao, J., Yang, W., & Yan, S. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2771408 · Cited by 175
A Recurrent Encoder-Decoder Network for Sequential Face Alignment
Peng, X., Feris, R., Wang, X., & Metaxas, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_3 · Cited by 153
Adaptive 3D Face Reconstruction from Unconstrained Photo Collections
Roth, J., Tong, Y., & Liu, X. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.455 · Cited by 143
Alexander, O., Mike Rogers, A., Lambeth, W., Chiang, M. J., & Debevec, P. E. (2009) · ACM SIGGRAPH Courses · doi:10.1145/1667239.1667251 · Cited by 133
Face reconstruction in the wild
Kemelmacher-Shlizerman, I., & Seitz, S. M. (2011) · International Conference on Computer Vision · doi:10.1109/iccv.2011.6126439 · Cited by 133
Face alignment in-the-wild: A Survey
Jin, X., & Tan, X. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.08.008 · Cited by 116
Offline Deformable Face Tracking in Arbitrary Videos
Chrysos, G. G., Antonakos, E., Zafeiriou, S. P., & Snape, P. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.126 · Cited by 102
gvnn: Neural Network Library for Geometric Computer Vision
Handa, A., Bloesch, M., Pătrăucean, V., Stent, S., McCormac, J., & Davison, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_9 · Cited by 100
Total Moving Face Reconstruction
Suwajanakorn, S., Kemelmacher-Shlizerman, I., & Seitz, S. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_52 · Cited by 93
Deep Lambertian NetworksPreprint
Tang, Y., Salakhutdinov, R., & Hinton, G. E. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6445 · Cited by 76
What Makes Tom Hanks Look Like Tom Hanks
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.450 · Cited by 76
Bulat, A., & Tzimiropoulos, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48881-3_43 · Cited by 74
Perspective-aware manipulation of portrait photos
Fried, O., Shechtman, E., Goldman, D. B., & Finkelstein, A. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925933 · Cited by 70
Convolutional Network for Attribute-driven and Identity-preserving Human Face GenerationPreprint
Li, M., Zuo, W., & Zhang, D. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.06434 · Cited by 53
Discriminative Deep Face Shape Model for Facial Point Detection
Wu, Y., & Ji, Q. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-014-0775-8 · Cited by 52
Inverting face embeddings with convolutional neural networksPreprint
Zhmoginov, A., & Sandler, M. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04189 · Cited by 52
DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild
Güler, R. A., Trigeorgis, G., Antonakos, E., Snape, P., Zafeiriou, S. P., & Kokkinos, I. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.280 · Cited by 39
Analysis-by-Synthesis by Learning to Invert Generative Black Boxes
Nair, V. B., Susskind, J., & Hinton, G. E. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87536-9_99 · Cited by 38
Deep Appearance Models: A Deep Boltzmann Machine Approach for Face Modeling
Duong, C. N., Luu, K., Quach, K. G., & Bui, T. D. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1113-3 · Cited by 32
On 3D face reconstruction via cascaded regression in shape space
Liu, F., Zeng, D., Li, J., & Zhao, Q. (2017) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700253 · Cited by 29
Deep Disentangled Representations for Volumetric Reconstruction
Grant, E., Kohli, P., & van Gerven, M. A. J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_22 · Cited by 27
Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., & Seitz, S. M. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_25 · Cited by 23
Background modeling for generative image models
Schönborn, S., Egger, B., Forster, A., & Vetter, T. R. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.01.008 · Cited by 13
Cited by
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285
Deep face recognition: A survey
Wang, M., & Deng, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.10.081 · Cited by 996
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_33 · Cited by 767
Metadata
- Authors
- Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 3735-3744
- OpenAlex
- W2604672468
- Cited by
- 343
- References
- 59
- FWCI
- 14.40
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.