The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression
Open accessAbstract
3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination. In general these methods require complex and inefficient pipelines for model building and fitting. In this work, we propose to address many of these limitations by training a Convolutional Neural Network (CNN) on an appropriate dataset consisting of 2D images and 3D facial models or scans. Our CNN works with just a single 2D facial image, does not require accurate alignment nor establishes dense correspondence between images, works for arbitrary facial poses and expressions, and can be used to reconstruct the whole 3D facial geometry (including the non-visible parts of the face) bypassing the construction (during training) and fitting (during testing) of a 3D Morphable Model. We achieve this via a simple CNN architecture that performs direct regression of a volumetric representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related task of facial landmark localization can be incorporated into the proposed framework and help improve reconstruction quality, especially for the cases of large poses and facial expressions. Code and models will be made available at http://aaronsplace.co.uk.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 3×
“In contrast, we process faces at fixed scale (assuming that this is provided by a face detector), but we build our CNN based on a state-of-the-art bottom-up top-down module [15] that allows analysing and combining CNN features at different resolutions for eventually making predictions at voxel level.”
MethodA morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
Efficient object localization using Convolutional Networks
Tompson, J., Goroshin, R., Jain, A., LeCun, Y., & Bregler, C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298664 · Cited by 1,365
A 3D Face Model for Pose and Illumination Invariant Face Recognition
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. (2009) · 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance · doi:10.1109/avss.2009.58 · Cited by 1,326
Face Alignment Across Large Poses: A 3D Solution
Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.23 · Cited by 1,137
Cited by
Deep learning for cellular image analysis
Moen, E., Bannon, D., Kudo, T., Graf, W., Covert, M., & Van Valen, D. (2019) · Nature Methods · doi:10.1038/s41592-019-0403-1 · Cited by 1,225
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_33 · Cited by 767
Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set
Deng, Y., Yang, J., Xu, S., Chen, D., Jia, Y., & Tong, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00038 · Cited by 763
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793
Learning an animatable detailed 3D face model from in-the-wild images
Feng, Y., Feng, H., Black, M. J., & Bolkart, T. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459936 · Cited by 660
Generating 3D Faces Using Convolutional Mesh Autoencoders
Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_43 · Cited by 560
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Kolotouros, N., Pavlakos, G., & Daniilidis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00463 · Cited by 548
Towards Fast, Accurate and Stable 3D Dense Face Alignment
Guo, J., Zhu, X., Yang, Y., Yang, F., Lei, Z., & Li, S. Z. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58529-7_10 · Cited by 462
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
Han, X.-F., Laga, H., & Bennamoun, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2954885 · Cited by 431
Shape-shifting structured lattices via multimaterial 4D printing
Boley, J. W., van Rees, W. M., Lissandrello, C., Horenstein, M. N., Truby, R. L., Kotikian, A., Lewis, J. A., & Mahadevan, L. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1908806116 · Cited by 420
BodyNet: Volumetric Inference of 3D Human Body Shapes
Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., & Schmid, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_2 · Cited by 385
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
Learning to Regress 3D Face Shape and Expression From an Image Without 3D Supervision
Sanyal, S., Bolkart, T., Feng, H., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00795 · Cited by 332
Unsupervised Training for 3D Morphable Model Regression
Genova, K., Cole, F., Maschinot, A., Sarna, A., Vlasic, D., & Freeman, W. T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00874 · Cited by 311
Uncertainty-Aware Mesh Decoder for High Fidelity 3D Face Reconstruction
Lee, G.-H., & Lee, S.-W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00614 · Cited by 70
Taati, B., Zhao, S., Ashraf, A. B., Asgarian, A., Browne, M. E., Prkachin, K. M., Mihailidis, A., & Hadjistavropoulos, T. (2019) · IEEE Access · doi:10.1109/access.2019.2900022 · Cited by 70
Metadata
- Authors
- Jackson, A. S., Bulat, A., Argyriou, V., & Tzimiropoulos, G.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1031-1039
- OpenAlex
- W2599226450
- Cited by
- 449
- References
- 28
- FWCI
- 17.96
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.