The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Recovering non-rigid 3D shape from image streams
Abstract
The paper addresses the problem of recovering 3D non-rigid shape models from image sequences. For example, given a video recording of a talking person, we would like to estimate a 3D model of the lips and the full face and its internal modes of variation. Many solutions that recover 3D shape from 2D image sequences have been proposed; these so-called structure-from-motion techniques usually assume that the 3D object is rigid. For example, C. Tomasi and T. Kanades' (1992) factorization technique is based on a rigid shape matrix, which produces a tracking matrix of rank 3 under orthographic projection. We propose a novel technique based on a non-rigid model, where the 3D shape in each frame is a linear combination of a set of basis shapes. Under this model, the tracking matrix is of higher rank, and can be factored in a three-step process to yield pose, configuration and shape. To the best of our knowledge, this is the first model free approach that can recover from single-view video sequences nonrigid shape models. We demonstrate this new algorithm on several video sequences. We were able to recover 3D non-rigid human face and animal models with high accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
Shi, J., & Tomasi (1994) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94 · doi:10.1109/cvpr.1994.323794 · Cited by 7,724
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
Shape and motion from image streams under orthography: a factorization method
Tomasi, C., & Kanade, T. (1992) · International Journal of Computer Vision · doi:10.1007/bf00129684 · Cited by 2,878
Synthesizing realistic facial expressions from photographs
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., & Salesin, D. H. (2006) · ACM SIGGRAPH 2006 Courses on - SIGGRAPH '06 · doi:10.1145/1185657.1185859 · Cited by 723
A Multibody Factorization Method for Independently Moving Objects
Costeira, J. P., & Kanade, T. (1998) · International Journal of Computer Vision · doi:10.1023/a:1008000628999 · Cited by 661
Synthesizing realistic facial expressions from photographs
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., & Salesin, D. H. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280825 · Cited by 462
Guenter, B., Grimm, C., Wood, D., Malvar, H., & Pighin, F. (2006) · ACM SIGGRAPH 2006 Courses on - SIGGRAPH '06 · doi:10.1145/1185657.1185858 · Cited by 387
Guenter, B., Grimm, C., Wood, D., Malvar, H., & Pighin, F. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280822 · Cited by 258
Resynthesizing facial animation through 3D model-based tracking
Pighin, F., Szeliski, R., & Salesin, D. H. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.791210 · Cited by 190
Cited by
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Simultaneous Localization, Mapping and Moving Object Tracking
Wang, C.-C., Thorpe, C., Thrun, S., Hebert, M., & Durrant-Whyte, H. (2007) · The International Journal of Robotics Research · doi:10.1177/0278364907081229 · Cited by 596
Real-time 3D model acquisition
Rusinkiewicz, S., Hall-Holt, O., & Levoy, M. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566600 · Cited by 576
Face transfer with multilinear models
Vlasic, D., Brand, M., Pfister, H., & Popović, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073209 · Cited by 556
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
Tome, D., Russell, C., & Agapito, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.603 · Cited by 504
Yan, J., & Pollefeys, M. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_8 · Cited by 489
Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors
Torresani, L., Hertzmann, A., & Bregler, C. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70752 · Cited by 456
Self-citationReal-time combined 2D+3D active appearance models
Xiao, J., Baker, S., Matthews, I., & Kanade, T. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315210 · Cited by 416
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
Visual SLAM and Structure from Motion in Dynamic Environments: A Survey
Saputra, M. R. U., Markham, A., & Trigoni, N. (2019) · ACM Computing Surveys · doi:10.1145/3177853 · Cited by 398
3D Menagerie: Modeling the 3D Shape and Pose of Animals
Zuffi, S., Kanazawa, A., Jacobs, D. W., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.586 · Cited by 381
Zhu, Z., Ng, D. W. H., Park, H. S., & McAlpine, M. C. (2020) · Nature Reviews Materials · doi:10.1038/s41578-020-00235-2 · Cited by 348
Damped Newton Algorithms for Matrix Factorization with Missing Data
Buchanan, A. M., & Fitzgibbon, A. W. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.118 · Cited by 299
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., et al. (2013) · Medical Image Analysis · doi:10.1016/j.media.2013.04.003 · Cited by 280
A Simple Prior-Free Method for Non-rigid Structure-from-Motion Factorization
Dai, Y., Li, H., & He, M. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0684-2 · Cited by 270
3D Shape Induction from 2D Views of Multiple Objects
Gadelha, M., Maji, S., & Wang, R. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00053 · Cited by 260
Category-specific object reconstruction from a single image
Kar, A., Tulsiani, S., Carreira, J., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298807 · Cited by 255
Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video
Garg, R., Roussos, A., & Agapito, L. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.168 · Cited by 210
Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
Wang, C., Lucey, S., Perazzi, F., & Wang, O. (2019) · 2019 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00046 · Cited by 103
Modeling deformable objects from a single depth camera
Liao, M., Zhang, Q., Wang, H., Yang, R., & Gong, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459161 · Cited by 101
Total Moving Face Reconstruction
Suwajanakorn, S., Kemelmacher-Shlizerman, I., & Seitz, S. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_52 · Cited by 93
Birds of a Feather: Capturing Avian Shape Models from Images
Wang, Y., Kolotouros, N., Daniilidis, K., & Badger, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01450 · Cited by 28
Metadata
- Authors
- Bregler, C., Hertzmann, A., & Biermann, H.
- Venue
- Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662)
- Publisher
- IEEE Comput. Soc
- Published
- 2002-11-07
- Volume
- 2
- Pages
- 690-696
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2124600577
- Cited by
- 754
- References
- 13
- FWCI
- 17.90
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.