The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Learning a model of facial shape and expression from 4D scans
Abstract
The field of 3D face modeling has a large gap between high-end and low-end methods. At the high end, the best facial animation is indistinguishable from real humans, but this comes at the cost of extensive manual labor. At the low end, face capture from consumer depth sensors relies on 3D face models that are not expressive enough to capture the variability in natural facial shape and expression. We seek a middle ground by learning a facial model from thousands of accurately aligned 3D scans. Our FLAME model (Faces Learned with an Articulated Model and Expressions) is designed to work with existing graphics software and be easy to fit to data. FLAME uses a linear shape space trained from 3800 scans of human heads. FLAME combines this linear shape space with an articulated jaw, neck, and eyeballs, pose-dependent corrective blendshapes, and additional global expression blendshapes. The pose and expression dependent articulations are learned from 4D face sequences in the D3DFACS dataset along with additional 4D sequences. We accurately register a template mesh to the scan sequences and make the D3DFACS registrations available for research purposes. In total the model is trained from over 33, 000 scans. FLAME is low-dimensional but more expressive than the FaceWarehouse model and the Basel Face Model. We compare FLAME to these models by fitting them to static 3D scans and 4D sequences using the same optimization method. FLAME is significantly more accurate and is available for research purposes (http://flame.is.tue.mpg.de).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2021) · A First Course in Statistical Programming with R · doi:10.1017/9781108993456.011 · Cited by 13,724
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Self-citationSupervised Descent Method and Its Applications to Face Alignment
Xiong, X., & De la Torre, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.75 · Cited by 1,896
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
A 3D Face Model for Pose and Illumination Invariant Face Recognition
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. (2009) · 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance · doi:10.1109/avss.2009.58 · Cited by 1,326
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171 · cited 5×
“Figure 15 shows the cumulative scan-to-mesh distance computed over all model-fits to the neutral BU-3DFE scans.”
InfluentialDeformation transfer for triangle meshes
Sumner, R. W., & Popović, J. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015736 · Cited by 1,052
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.249 · Cited by 1,023
The space of human body shapes: reconstruction and parameterization from range scans
Allen, B., Curless, B., & Popović, Z. (2003) · ACM SIGGRAPH 2003 Papers · doi:10.1145/1201775.882311 · Cited by 913
Optimal Step Nonrigid ICP Algorithms for Surface Registration
Amberg, B., Romdhani, S., & Vetter, T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383165 · Cited by 746
Numerical Geometry of Non-Rigid Shapes
Bronstein, A. M., Bronstein, M. M., & Kimmel, R. (2009) · Monographs in Computer Science · doi:10.1007/978-0-387-73301-2 · Cited by 734
The space of human body shapes: reconstruction and parameterization from range scans
Allen, B., Curless, B., & Popović, Z. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882311 · Cited by 723
British Machine Vision Conference 2000
Mirmehdi, M. (2002) · Image and Vision Computing · doi:10.1016/s0262-8856(02)00048-3 · Cited by 716
FAUST: Dataset and Evaluation for 3D Mesh Registration
Bogo, F., Romero, J., Loper, M., & Black, M. J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.491 · Cited by 689
Self-citationInteractive multi-resolution modeling on arbitrary meshes
Kobbelt, L., Campagna, S., Vorsatz, J., & Seidel, H.-P. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280831 · Cited by 655
Statistical methods for tomographic image reconstruction
Geman, S., & McClure, D. E. (1987) · Medical Entomology and Zoology · Cited by 579
Face transfer with multilinear models
Vlasic, D., Brand, M., Pfister, H., & Popović, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073209 · Cited by 556
Realtime performance-based facial animation
Weise, T., Bouaziz, S., Li, H., & Pauly, M. (2011) · ACM SIGGRAPH 2011 papers · doi:10.1145/1964921.1964972 · Cited by 460
Self-citationOpenDR: An Approximate Differentiable Renderer
Loper, M. M., & Black, M. J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_11 · Cited by 441
Self-citationA Statistical Model of Human Pose and Body Shape
Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., & Seidel, H. ‐. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01373.x · Cited by 390
Real-time expression transfer for facial reenactment
Thies, J., Zollhöfer, M., Nießner, M., Valgaerts, L., Stamminger, M., & Theobalt, C. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818056 · Cited by 369
Realtime performance-based facial animation
Weise, T., Bouaziz, S., Li, H., & Pauly, M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964972 · Cited by 356
Self-citationA 3D Morphable Model Learnt from 10,000 Faces
Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A., & Dunaway, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.598 · Cited by 353
Large Scale 3D Morphable Models
Booth, J., Roussos, A., Ponniah, A., Dunaway, D., & Zafeiriou, S. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-017-1009-7 · Cited by 329
Realtime facial animation with on-the-fly correctives
Li, H., Yu, J., Ye, Y., & Bregler, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462019 · Cited by 282
Self-citationOnline modeling for realtime facial animation
Bouaziz, S., Wang, Y., & Pauly, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461976 · Cited by 273
Robinette, K. M., Blackwell, S., Daanen, H., Boehmer, M., & Fleming, S. (2002) · doi:10.21236/ada406704 · Cited by 260
Reconstruction of Personalized 3D Face Rigs from Monocular Video
Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2890493 · Cited by 260
Dynamic 3D avatar creation from hand-held video input
Ichim, A. E., Bouaziz, S., & Pauly, M. (2015) · ACM Transactions on Graphics · doi:10.1145/2766974 · Cited by 249
High-quality passive facial performance capture using anchor frames
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., & Gross, M. (2011) · ACM SIGGRAPH 2011 papers · doi:10.1145/1964921.1964970 · Cited by 238
High-quality passive facial performance capture using anchor frames
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., & Gross, M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964970 · Cited by 223
Real-time high-fidelity facial performance capture
Cao, C., Bradley, D., Zhou, K., & Beeler, T. (2015) · ACM Transactions on Graphics · doi:10.1145/2766943 · Cited by 217
Building statistical shape spaces for 3D human modeling
Pishchulin, L., Wuhrer, S., Helten, T., Theobalt, C., & Schiele, B. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.02.018 · Cited by 210
Reconstructing detailed dynamic face geometry from monocular video
Garrido, P., Valgaert, L., Wu, C., & Theobalt, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508380 · Cited by 204
Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences
Bogo, F., Black, M. J., Loper, M., & Romero, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.265 · Cited by 200
Self-citationExpression invariant 3D face recognition with a Morphable Model
Amberg, B., Knothe, R., & Vetter, T. (2008) · 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition · doi:10.1109/afgr.2008.4813376 · Cited by 189
Li, H., Weise, T., & Pauly, M. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778769 · Cited by 188
Self-citationCoregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape
Hirshberg, D. A., Loper, M., Rachlin, E., & Black, M. J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_18 · Cited by 173
Self-citationCosker, D. P., Krumhuber, E. G., & Hilton, A. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126510 · Cited by 162
Sparse localized deformation components
Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M. A., & Theobalt, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508417 · Cited by 157
Alexander, O., Mike Rogers, A., Lambeth, W., Chiang, M. J., & Debevec, P. E. (2009) · ACM SIGGRAPH Courses · doi:10.1145/1667239.1667251 · Cited by 133
Face reconstruction in the wild
Kemelmacher-Shlizerman, I., & Seitz, S. M. (2011) · International Conference on Computer Vision · doi:10.1109/iccv.2011.6126439 · Cited by 133
Expression flow for 3D-aware face component transfer
Yang, F. H., Wang, J., Shechtman, E., Bourdev, L. D., & Metaxas, D. (2011) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1964921.1964955 · Cited by 130
Expression flow for 3D-aware face component transfer
Yang, F. H., Wang, J., Shechtman, E., Bourdev, L. D., & Metaxas, D. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964955 · Cited by 128
Automatic acquisition of high-fidelity facial performances using monocular videos
Shi, F., Wu, H., Tong, X., & Chai, J. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661290 · Cited by 127
Allen, B. L., Curless, B., Popović, Z., & Hertzmann, A. (2006) · Symposium on Computer Animation · doi:10.5555/1218064.1218084 · Cited by 116
Statistical Models of Shape: Optimisation and Evaluation
Davies, R. H., Twining, C., & Taylor, C. (2008) · CERN Document Server (European Organization for Nuclear Research) · Cited by 114
An anatomically-constrained local deformation model for monocular face capture
Wu, C., Bradley, D., Groß, M. H., & Beeler, T. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925882 · Cited by 108
Total Moving Face Reconstruction
Suwajanakorn, S., Kemelmacher-Shlizerman, I., & Seitz, S. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_52 · Cited by 93
What Makes Tom Hanks Look Like Tom Hanks
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.450 · Cited by 84
A Groupwise Multilinear Correspondence Optimization for 3D Faces
Bolkart, T., & Wuhrer, S. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.411 · Cited by 72
Self-citationFully automatic expression-invariant face correspondence
Salazar, A., Wuhrer, S., Shu, C., & Prieto, F. (2013) · Machine Vision and Applications · doi:10.1007/s00138-013-0579-9 · Cited by 57
Rigid stabilization of facial expressions
Beeler, T., & Bradley, D. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601182 · Cited by 54
Ferrari, C., Lisanti, G., Berretti, S., & Bimbo, A. D. (2015) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2015.63 · Cited by 40
Lightweight wrinkle synthesis for 3D facial modeling and animation
Jun, L., Xu, W., Cheng, Z., Xu, K., & Klein, R. (2014) · Computer-Aided Design · doi:10.1016/j.cad.2014.08.016 · Cited by 23
A Practical System for Modelling Body Shapes from Single View Measurements
Chen, Y., Robertson, D., & Cipolla, R. (2011) · British Machine Vision Conference (BMVC) · doi:10.5244/c.25.82 · Cited by 12
Multilinear Wavelets: A Statistical Shape Space for Human FacesPreprint
Brunton, A., Bolkart, T., & Wuhrer, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.2818 · Cited by 10
Self-citationA 3D Morphable Model of the Eye Region
Wood, E., Baltrušaitis, T., Morency, L., Robinson, P., & Bulling, A. (2016) · Eurographics · doi:10.2312/egp.20161054 · Cited by 8
Cited by
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationAMASS: Archive of Motion Capture As Surface Shapes
Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00554 · Cited by 1,323
Self-citationGenerating 3D Faces Using Convolutional Mesh Autoencoders
Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_43 · Cited by 560
Self-citationFastNeRF: High-Fidelity Neural Rendering at 200FPS
Garbin, S. J., Kowalski, M., Johnson, M., Shotton, J., & Valentin, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01408 · Cited by 541
Neural Fields in Visual Computing and Beyond
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., sitzmann, V., & Sridhar, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14505 · Cited by 484
Metadata
- Authors
- Li, T., Bolkart, T., Black, M. J., Li, H., & Romero, J.
- Venue
- ACM Transactions on Graphics
- Published
- 2017-11-20
- Volume
- 36
- Issue
- 6
- Pages
- 1-17
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2769666294
- Cited by
- 1,223
- References
- 61
- FWCI
- 14.49
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.