The DOI matches its Crossref record; 159 of the 159 references it cites were found in a registry or an index.
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications
Abstract
The computer graphics and vision communities have dedicated long standing efforts in building computerized tools for reconstructing, tracking, and analyzing human faces based on visual input. Over the past years rapid progress has been made, which led to novel and powerful algorithms that obtain impressive results even in the very challenging case of reconstruction from a single RGB or RGB‐D camera. The range of applications is vast and steadily growing as these technologies are further improving in speed, accuracy, and ease of use. Motivated by this rapid progress, this state‐of‐the‐art report summarizes recent trends in monocular facial performance capture and discusses its applications, which range from performance‐based animation to real‐time facial reenactment. We focus our discussion on methods where the central task is to recover and track a three dimensional model of the human face using optimization‐based reconstruction algorithms. We provide an in‐depth overview of the underlying concepts of real‐world image formation, and we discuss common assumptions and simplifications that make these algorithms practical. In addition, we extensively cover the priors that are used to better constrain the under‐constrained monocular reconstruction problem, and discuss the optimization techniques that are employed to recover dense, photo‐geometric 3D face models from monocular 2D data. Finally, we discuss a variety of use cases for the reviewed algorithms in the context of motion capture, facial animation, as well as image and video editing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
Marquardt, D. W. (1963) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0111030 · Cited by 30,728
A flexible new technique for camera calibration
Zhang, Z. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.888718 · Cited by 15,004
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
A method for the solution of certain non-linear problems in least squares
Levenberg, K. (1944) · Quarterly of Applied Mathematics · doi:10.1090/qam/10666 · Cited by 12,359
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Face recognition: A literature survey
Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003) · ACM Computing Surveys · doi:10.1145/954339.954342 · Cited by 6,163
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
Computer Vision: A Modern Approach
Forsyth, D. A., & Ponce, J. (2002) · Cited by 3,739
Numerical Methods for Least Squares Problems
Björck, Å. (1996) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971484 · Cited by 3,257
Object modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583
The psychology of computer vision
Winston, P. H. (1976) · Pattern Recognition · doi:10.1016/0031-3203(76)90020-0 · Cited by 2,569
Face detection, pose estimation, and landmark localization in the wild
Zhu, X., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248014 · Cited by 2,120
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & De la Torre, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.75 · Cited by 1,896
Zhang, R., Tsai, P.-S., Cryer, J. E., & Shah, M. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.784284 · Cited by 1,731
In the Eye of the Beholder: A Survey of Models for Eyes and Gaze
Hansen, D. W., & Ji, Q. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.30 · Cited by 1,546
Active Appearance Models Revisited
Matthews, I., & Baker, S. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029666.37597.d3 · Cited by 1,535
Learning a model of facial shape and expression from 4D scans
Li, T., Bolkart, T., Black, M. J., Li, H., & Romero, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130813 · Cited by 1,223
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057
Deformation transfer for triangle meshes
Sumner, R. W., & Popović, J. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015736 · Cited by 1,052
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.249 · Cited by 1,023
2D and 3D face recognition: A survey
Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007) · Pattern Recognition Letters · doi:10.1016/j.patrec.2006.12.018 · Cited by 876
A practical model for subsurface light transport
Jensen, H. W., Marschner, S. R., Levoy, M., & Hanrahan, P. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383319 · Cited by 823
Deformable Model Fitting by Regularized Landmark Mean-Shift
Saragih, J. M., Lucey, S., & Cohn, J. F. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0380-4 · Cited by 787
An efficient representation for irradiance environment maps
Ramamoorthi, R., & Hanrahan, P. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383317 · Cited by 769
Feature Detection and Tracking with Constrained Local Models
Cristinacce, D., & Cootes, T. F. (2006) · Procedings of the British Machine Vision Conference 2006 · doi:10.5244/c.20.95 · Cited by 718
Video Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669
A signal-processing framework for inverse rendering
Ramamoorthi, R., & Hanrahan, P. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383271 · Cited by 577
Robust Discriminative Response Map Fitting with Constrained Local Models
Asthana, A., Zafeiriou, S., Cheng, S., & Pantic, M. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.442 · Cited by 557
Face transfer with multilinear models
Vlasic, D., Brand, M., Pfister, H., & Popović, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073209 · Cited by 556
Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
Tran, A. T., Hassner, T., Masi, I., & Medioni, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.163 · Cited by 503
Spacetime faces: high resolution capture for modeling and animation
Zhang, L., Snavely, N., Curless, B., & Seitz, S. M. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015759 · Cited by 493
A survey on face detection in the wild: Past, present and future
Zafeiriou, S., Zhang, C., & Zhang, Z. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.03.015 · Cited by 458
Audio-driven facial animation by joint end-to-end learning of pose and emotion
Karras, T., Aila, T., Laine, S., Herva, A., & Lehtinen, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073658 · Cited by 434
Real-time combined 2D+3D active appearance models
Xiao, J., Baker, S., Matthews, I., & Kanade, T. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315210 · Cited by 416
Reanimating Faces in Images and Video
Blanz, V., Basso, C., Poggio, T., & Vetter, T. (2003) · Computer Graphics Forum · doi:10.1111/1467-8659.t01-1-00712 · Cited by 387
Spectral Geometry Processing with Manifold Harmonics
Vallet, B., & Lévy, B. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01122.x · Cited by 386
Real-time expression transfer for facial reenactment
Thies, J., Zollhöfer, M., Nießner, M., Valgaerts, L., Stamminger, M., & Theobalt, C. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818056 · Cited by 369
Self-citationLipreading and audio-visual speech perception
Summerfield, Q. (1992) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1992.0009 · Cited by 358
Static and dynamic 3D facial expression recognition: A comprehensive survey
Sandbach, G., Zafeiriou, S., Pantic, M., & Yin, L. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.005 · Cited by 357
Realtime performance-based facial animation
Weise, T., Bouaziz, S., Li, H., & Pauly, M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964972 · Cited by 356
Displaced dynamic expression regression for real-time facial tracking and animation
Cao, C., Hou, Q., & Zhou, K. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601204 · Cited by 356
A 3D Morphable Model Learnt from 10,000 Faces
Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A., & Dunaway, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.598 · Cited by 353
3D shape regression for real-time facial animation
Cao, C., Weng, Y., Lin, S., & Zhou, K. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462012 · Cited by 348
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.401 · Cited by 343
Self-citationLearning Detailed Face Reconstruction from a Single Image
Richardson, E., Sela, M., Or-El, R., & Kimmel, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.589 · Cited by 336
3-D motion estimation in model-based facial image coding
Li, H., Roivainen, P., & Forchheimer, R. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.216724 · Cited by 335
Face swapping: automatically replacing faces in photographs
Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., & Nayar, S. K. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360638 · Cited by 333
3D Constrained Local Model for rigid and non-rigid facial tracking
Baltrusaitis, T., Robinson, P., & Morency, L. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247980 · Cited by 313
Automatic determination of facial muscle activations from sparse motion capture marker data
Sifakis, E., Neverov, I., & Fedkiw, R. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073208 · Cited by 312
3D Face Reconstruction by Learning from Synthetic Data
Richardson, E., Sela, M., & Kimmel, R. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.56 · Cited by 309
A deep learning approach for generalized speech animation
Taylor, S., Kim, T., Yue, Y., Mahler, M., Krahe, J., Rodriguez, A. G., Hodgins, J., & Matthews, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073699 · Cited by 303
Trainable videorealistic speech animation
Ezzat, T., Geiger, G., & Poggio, T. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566594 · Cited by 301
Optical Flow Constraints on Deformable Models with Applications to Face Tracking
DeCarlo, D., & Metaxas, D. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008122917811 · Cited by 299
Realtime facial animation with on-the-fly correctives
Li, H., Yu, J., Ye, Y., & Bregler, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462019 · Cited by 282
Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation
Sela, M., Richardson, E., & Kimmel, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.175 · Cited by 280
Online modeling for realtime facial animation
Bouaziz, S., Wang, Y., & Pauly, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461976 · Cited by 273
Reconstruction of Personalized 3D Face Rigs from Monocular Video
Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2890493 · Cited by 260
Self-citationGuenter, B., Grimm, C., Wood, D., Malvar, H., & Pighin, F. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280822 · Cited by 258
Dynamic 3D avatar creation from hand-held video input
Ichim, A. E., Bouaziz, S., & Pauly, M. (2015) · ACM Transactions on Graphics · doi:10.1145/2766974 · Cited by 249
Expressive expression mapping with ratio images
Liu, Z., Shan, Y., & Zhang, Z. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383289 · Cited by 240
High resolution passive facial performance capture
Bradley, D., Heidrich, W., Popa, T., & Sheffer, A. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778778 · Cited by 233
Self-citationBlanz, V., Scherbaum, K., Vetter, T., & Seidel, H. (2004) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2004.00799.x · Cited by 226
Noh, J.-Y., & Neumann, U. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383290 · Cited by 224
High-quality passive facial performance capture using anchor frames
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., & Gross, M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964970 · Cited by 223
Self-citationReal-time high-fidelity facial performance capture
Cao, C., Bradley, D., Zhou, K., & Beeler, T. (2015) · ACM Transactions on Graphics · doi:10.1145/2766943 · Cited by 217
Self-citationDeCarlo, D., & Metaxas, D. (1996) · Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1996.517079 · Cited by 205
Reconstructing detailed dynamic face geometry from monocular video
Garrido, P., Valgaert, L., Wu, C., & Theobalt, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508380 · Cited by 204
Self-citationFace/Off: live facial puppetry
Weise, T., Li, H., Van Gool, L., & Pauly, M. (2009) · Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation · doi:10.1145/1599470.1599472 · Cited by 194
Resynthesizing facial animation through 3D model-based tracking
Pighin, F., Szeliski, R., & Salesin, D. H. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.791210 · Cited by 190
Li, H., Weise, T., & Pauly, M. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778769 · Cited by 188
Averbuch-Elor, H., Cohen-Or, D., Kopf, J., & Cohen, M. F. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130818 · Cited by 187
Multiview face capture using polarized spherical gradient illumination
Ghosh, A., Fyffe, G., Tunwattanapong, B., Busch, J., Yu, X., & Debevec, P. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024163 · Cited by 185
Garrido, P., Valgaerts, L., Rehmsen, O., Thormaehlen, T., Perez, P., & Theobalt, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.537 · Cited by 172
Self-citationReal-time facial animation with image-based dynamic avatars
Cao, C., Wu, H., Weng, Y., Shao, T., & Zhou, K. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925873 · Cited by 172
Camera Models and Fundamental Concepts Used in Geometric Computer Vision
Sturm, P., Ramalingam, S., Tardif, J.-P., Gasparini, S., & Barreto, J. (2011) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000023 · Cited by 166
Visemes Observed by Hearing-Impaired and Normal-Hearing Adult Viewers
Owens, E., & Blazek, B. (1985) · Journal of Speech, Language, and Hearing Research · doi:10.1044/jshr.2803.381 · Cited by 163
The Digital Emily Project: Achieving a Photorealistic Digital Actor
Alexander, O., Rogers, M., Lambeth, W., Chiang, J.-Y., Ma, W.-C., Wang, C.-C., & Debevec, P. (2010) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2010.65 · Cited by 162
Avatar digitization from a single image for real-time rendering
Hu, L., Saito, S., Wei, L., Nagano, K., Seo, J., Fursund, J., Sadeghi, I., Sun, C., Chen, Y.-C., & Li, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.31310887 · Cited by 161
Sparse localized deformation components
Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M. A., & Theobalt, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508417 · Cited by 157
Self-citationPerformance-driven facial animation
Williams, L. R. (1990) · ACM SIGGRAPH Computer Graphics · doi:10.1145/97880.97906 · Cited by 154
Lightweight binocular facial performance capture under uncontrolled lighting
Valgaerts, L., Wu, C., Bruhn, A., Seidel, H., & Theobalt, C. (2012) · ACM Transactions on Graphics · doi:10.1145/2366145.2366206 · Cited by 143
Self-citationHigh-fidelity facial and speech animation for VR HMDs
Olszewski, K., Lim, J. J., Saito, S., & Li, H. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980252 · Cited by 140
Analyzing facial expressions for virtual conferencing
Eisert, P., & Girod, B. (1998) · IEEE Computer Graphics and Applications · doi:10.1109/38.708562 · Cited by 137
Facial performance synthesis using deformation-driven polynomial displacement maps
Ma, W., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., & Debevec, P. E. (2008) · ACM Transactions on Graphics · doi:10.1145/1409060.1409074 · Cited by 136
High-quality shape from multi-view stereo and shading under general illumination
Wu, C., Wilburn, B. S., Matsushita, Y., & Theobalt, C. (2011) · doi:10.1109/cvpr.2011.5995388 · Cited by 136
Self-citationTrainable videorealistic speech animation
Ezzat, T., Geiger, G., & Poggio, T. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566594 · Cited by 135
Modeling, tracking and interactive animation of faces and heads//using input from video
Essa, I. A., Basu, S., Darrell, T., & Pentland, A. (2002) · Computer Animation (CA) · doi:10.1109/ca.1996.540489 · Cited by 134
High Resolution Acquisition, Learning and Transfer of Dynamic 3‐D Facial Expressions
Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., & Huang, P. S. (2004) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2004.00800.x · Cited by 133
Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., & Durand, F. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360629 · Cited by 131
Single-view hair modeling using a hairstyle database
Hu, L., Ma, C., Luo, L., & Li, H. (2015) · ACM Transactions on Graphics · doi:10.1145/2766931 · Cited by 130
VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track
Garrido, P., Valgaerts, L., Sarmadi, H., Steiner, I., Varanasi, K., Pérez, P., & Theobalt, C. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12552 · Cited by 130
Self-citationMulti-scale capture of facial geometry and motion
Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M. Á., Pfister, H., & Groß, M. H. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276419 · Cited by 129
Dale, K. M., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., & Pfister, H. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024164 · Cited by 128
Automatic acquisition of high-fidelity facial performances using monocular videos
Shi, F., Wu, H., Tong, X., & Chai, J. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661290 · Cited by 127
Real-Time Facial Segmentation and Performance Capture from RGB Input
Saito, S., Li, T., & Li, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_15 · Cited by 127
AutoHair: fully automatic hair modeling from a single image
Chai, M., Shao, T., Wu, H., Weng, Y., & Zhou, K. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925961 · Cited by 118
A Variational Approach to Video Registration with Subspace Constraints
Garg, R., Roussos, A., & Agapito, L. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0607-7 · Cited by 115
Ichim, A.-E., Kadleček, P., Kavan, L., & Pauly, M. V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073664 · Cited by 114
Unconstrained realtime facial performance capture
Hsieh, P.-L., Ma, C., Yu, J., & Li, H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298776 · Cited by 112
Computer‐Suggested Facial Makeup
Scherbaum, K., Ritschel, T., Hullin, M. B., Thormählen, T., Blanz, V., & Seidel, H. (2011) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2011.01874.x · Cited by 105
Makeup Lamps: Live Augmentation of Human Faces via Projection
Bermano, A. H., Billeter, M., Iwai, D., & Grundhöfer, A. (2017) · Computer Graphics Forum · doi:10.1111/cgf.13128 · Cited by 104
Production-level facial performance capture using deep convolutional neural networks
Laine, S., Karras, T., Aila, T., Herva, A., Saito, S., Yu, R., Li, H., & Lehtinen, J. (2017) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/3099564.3099581 · Cited by 102
A 3D Morphable Eye Region Model for Gaze Estimation
Wood, E., Baltrušaitis, T., Morency, L., Robinson, P., & Bulling, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_18 · Cited by 98
High-quality hair modeling from a single portrait photo
Chai, M., Luo, L., Sunkavalli, K., Carr, N., Hadap, S., & Zhou, K. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818112 · Cited by 96
Driving High-Resolution Facial Scans with Video Performance Capture
Fyffe, G., Jones, A., Alexander, O., Ichikari, R., & Debevec, P. E. (2014) · ACM Transactions on Graphics · doi:10.1145/2638549 · Cited by 94
Total Moving Face Reconstruction
Suwajanakorn, S., Kemelmacher-Shlizerman, I., & Seitz, S. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_52 · Cited by 93
Expressive Visual Text-to-Speech Using Active Appearance Models
Anderson, R., Stenger, B., Wan, V., & Cipolla, R. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.434 · Cited by 86
An anatomically-constrained local deformation model for monocular face capture
Wu, C., Bradley, D., Groß, M. H., & Beeler, T. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925882 · Cited by 86
Self-citationSpacetime expression cloning for blendshapes
Seol, Y., Lewis, J. P., Seo, J., Choi, B., Anjyo, K., & Noh, J. (2012) · ACM Transactions on Graphics · doi:10.1145/2159516.2159519 · Cited by 85
Multilinear Wavelets: A Statistical Shape Space for Human Faces
Brunton, A., Bolkart, T., & Wuhrer, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_20 · Cited by 82
Bérard, P., Bradley, D., Nitti, M., Beeler, T., & Groß, M. H. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661285 · Cited by 81
Self-citationKemelmacher-Shlizerman, I. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925871 · Cited by 79
On-set performance capture of multiple actors with a stereo camera
Wu, C., Stoll, C., Valgaerts, L., & Theobalt, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508418 · Cited by 77
Self-citationFlexible flow for 3D nonrigid tracking and shape recovery
Brand, M., & Bhotika, R. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2001.990492 · Cited by 77
Model-based teeth reconstruction
Wu, C., Bradley, D., Garrido, P., Zollhöfer, M., Theobalt, C., Groß, M. H., & Beeler, T. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980233 · Cited by 77
Self-citationWhat Makes Tom Hanks Look Like Tom Hanks
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.450 · Cited by 76
Internet Based Morphable Model
Kemelmacher-Shlizerman, I. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.404 · Cited by 70
Interactive region-based linear 3D face models
Tena, J. R., la Torre, F. D., & Matthews, I. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964971 · Cited by 69
Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition
Huang, H., Chai, J., Tong, X., & Wu, H. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964969 · Cited by 66
Visyllable Based Speech Animation
Kshirsagar, S., & Magnenat‐Thalmann, N. (2003) · Computer Graphics Forum · doi:10.1111/1467-8659.t01-2-00711 · Cited by 65
Lightweight eye capture using a parametric model
Bérard, P., Bradley, D., Groß, M. H., & Beeler, T. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925962 · Cited by 59
Self-citationFacial retargeting with automatic range of motion alignment
Ribera, R. B. I., Zell, E., Lewis, J. P., Noh, J., & Botsch, M. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073674 · Cited by 59
Video-audio driven real-time facial animation
Liu, Y., Xu, F., Chai, J., Tong, X., Wang, L., & Huo, Q. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818122 · Cited by 56
Temporal upsampling of performance geometry using photometric alignment
Wilson, C. A., Ghosh, A., Peers, P., Chiang, J.-Y., Busch, J., & Debevec, P. E. (2010) · ACM Transactions on Graphics · doi:10.1145/1731047.1731055 · Cited by 56
Real-time speech motion synthesis from recorded motions
Cao, Y., Faloutsos, P., Kohler, E., & Pighin, F. P. (2004) · Computer animation and simulation · doi:10.1145/1028523.1028570 · Cited by 56
Hu, L., Ma, C., Luo, L., Wei, L., & Li, H. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661254 · Cited by 55
Recent Advances in Facial Appearance Capture
Klehm, O., Rousselle, F., Papas, M., Bradley, D., Hery, C., Bickel, B., Jarosz, W., & Beeler, T. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12594 · Cited by 54
Self-citationAccurate and Robust 3D Facial Capture Using a Single RGBD Camera
Chen, Y., Wu, H., Shi, F., Tong, X., & Chai, J. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.449 · Cited by 54
Transferable videorealistic speech animation
Chang, Y., & Ezzat, T. (2005) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/1073368.1073388 · Cited by 51
High fidelity facial animation capture and retargeting with contours
Bhat, K. S., Goldenthal, R., Ye, Y., Mallet, R., & Koperwas, M. (2013) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/2485895.2485915 · Cited by 50
Accurate visible speech synthesis based on concatenating variable length motion capture data
Ma, J., Cole, R. A., Pellom, B. L., Ward, W., & Wise, B. W. (2006) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2006.18 · Cited by 50
Realtime 3D eye gaze animation using a single RGB camera
Wang, C., Shi, F., Xia, S., & Chai, J. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925947 · Cited by 50
FaceForge: Markerless Non-Rigid Face Multi-Projection Mapping
Siegl, C., Lange, V., Stamminger, M., Bauer, F., & Thies, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2734428 · Cited by 50
Self-citationDetailed spatio-temporal reconstruction of eyelids
Bermano, A. H., Beeler, T., Kozlov, Y., Bradley, D., Bickel, B., & Groß, M. H. (2015) · ACM Transactions on Graphics · doi:10.1145/2766924 · Cited by 49
Self-citationRigid stabilization of facial expressions
Beeler, T., & Bradley, D. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601182 · Cited by 49
Self-citationMulti-view hair capture using orientation fields
Luo, L., Li, H., Paris, S., Weise, T., Pauly, M. V., & Rusinkiewicz, S. M. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247838 · Cited by 47
Dense 3D motion capture for human faces
Furukawa, Y., & Ponce, J. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206868 · Cited by 46
Fully automatic generation of anatomical face simulation models
Cong, M., Bao, M., E, J. L., Bhat, K. S., & Fedkiw, R. P. (2015) · ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA) · doi:10.1145/2786784.2786786 · Cited by 45
Facial performance enhancement using dynamic shape space analysis
Bermano, A. H., Bradley, D., Beeler, T., Zünd, F., Nowrouzezahrai, D., Baran, I., Sorkine‐Hornung, O., Pfister, H., Sumner, R. W., Bickel, B., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2546276 · Cited by 44
Self-citationKemelmacher-Shlizerman, I., Shechtman, E., Garg, R. D., & Seitz, S. M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964956 · Cited by 44
Corrective 3D reconstruction of lips from monocular video
Garrido, P., Zollhöfer, M., Wu, C., Bradley, D., Pérez, P., Beeler, T., & Theobalt, C. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982419 · Cited by 42
Self-citationA Generic Framework for Efficient 2-D and 3-D Facial Expression Analogy
Song, M., Dong, Z., Theobalt, C., Wang, H., Liu, Z., & Seidel, H. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2007.906591 · Cited by 41
Self-citationAlexander, O., Fyffe, G., Busch, J., Yu, X., Ichikari, R., Jones, A., Debevec, P. E., Jiménez, J., Danvoye, É., Antionazzi, B., et al. (2013) · ACM SIGGRAPH Posters · doi:10.1145/2503385.2503387 · Cited by 37
Modern techniques and applications for real-time non-rigid registration
Bouaziz, S., Tagliasacchi, A., Li, H., & Pauly, M. V. (2016) · SIGGRAPH Asia Courses · doi:10.1145/2988458.2988490 · Cited by 37
A Data-Driven Approach for Facial Expression Retargeting in Video
Li, K., Dai, Q., Wang, R., Liu, Y., Xu, F., & Wang, J. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2293064 · Cited by 36
Realistic facial expression synthesis for an image-based talking head
Liu, K., & Östermann, J. (2011) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2011.6011835 · Cited by 32
Real-time avatar animation from a single image
Saragih, J., Lucey, S., & Cohn, J. F. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771400 · Cited by 32
Head Reconstruction from Internet Photos
Liang, S. Y., Shapiro, L. G., & Kemelmacher-Shlizerman, I. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_23 · Cited by 26
Real-time 3D eyelids tracking from semantic edges
Quan, W., Xu, F., Lu, M., & Yong, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130837 · Cited by 25
Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., & Seitz, S. M. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15549-9_25 · Cited by 23
Lightweight wrinkle synthesis for 3D facial modeling and animation
Jun, L., Xu, W., Cheng, Z., Xu, K., & Klein, R. (2014) · Computer-Aided Design · doi:10.1016/j.cad.2014.08.016 · Cited by 22
Synthetic Prior Design for Real-Time Face Tracking
McDonagh, S., Klaudiny, M., Bradley, D., Beeler, T., Matthews, I., & Mitchell, K. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.72 · Cited by 17
Self-citationFaceDirector: Continuous Control of Facial Performance in Video
Malleson, C., Bazin, J., Wang, O., Bradley, D., Beeler, T., Hilton, A., & Sorkine‐Hornung, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.453 · Cited by 15
Self-citationA hybrid approach to facial rigging
Komorowski, D., Melapudi, V., Mortillaro, D., & Lee, G. S. (2010) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1899950.1899992 · Cited by 10
Carnival—Combining Speech Technology and Computer Animation
Berger, M., Hofer, G., & Shimodaira, H. (2011) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2011.71 · Cited by 9
Detail-Preserving Controllable Deformation from Sparse Examples
Huang, H., Yin, K., Zhao, L., Qi, Y., Yu, Y., & Tong, X. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.88 · Cited by 7
Cited by
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
Self-citationExpressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824
Self-citationOn the Detection of Digital Face Manipulation
Dang, H., Liu, F., Stehouwer, J., Liu, X., & Jain, A. K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00582 · Cited by 764
Metadata
- Authors
- Zollhöfer, M., Thies, J., Garrido, P., Bradley, D., Beeler, T., Pérez, P., Stamminger, M., Nießner, M., & Theobalt, C.
- Venue
- Computer Graphics Forum
- Publisher
- Wiley
- Published
- 2018-05
- Volume
- 37
- Issue
- 2
- Pages
- 523-550
- ISSN
- 0167-7055 (ISSN-L)
- 1467-8659
- OpenAlex
- W2806379360
- Cited by
- 307
- References
- 159
- FWCI
- 17.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.