The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Abstract
This work addresses the problem of estimating the full body 3D human pose and shape from a single color image. This is a task where iterative optimization-based solutions have typically prevailed, while Convolutional Networks (ConvNets) have suffered because of the lack of training data and their low resolution 3D predictions. Our work aims to bridge this gap and proposes an efficient and effective direct prediction method based on ConvNets. Central part to our approach is the incorporation of a parametric statistical body shape model (SMPL) within our end-to-end framework. This allows us to get very detailed 3D mesh results, while requiring estimation only of a small number of parameters, making it friendly for direct network prediction. Interestingly, we demonstrate that these parameters can be predicted reliably only from 2D keypoints and masks. These are typical outputs of generic 2D human analysis ConvNets, allowing us to relax the massive requirement that images with 3D shape ground truth are available for training. Simultaneously, by maintaining differentiability, at training time we generate the 3D mesh from the estimated parameters and optimize explicitly for the surface using a 3D per-vertex loss. Finally, a differentiable renderer is employed to project the 3D mesh to the image, which enables further refinement of the network, by optimizing for the consistency of the projection with 2D annotations (i.e., 2D keypoints or masks). The proposed approach outperforms previous baselines on this task and offers an attractive solution for direct prediction of3D shape from a single color image.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
OctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.701 · Cited by 1,589
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
A Simple Yet Effective Baseline for 3d Human Pose Estimation
Martinez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,462
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00064 · Cited by 1,142
DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation
Pishchulin, L., Insafutdinov, E., Tang, S., Andres, B., Andriluka, M., Gehler, P., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.533 · Cited by 1,051
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 915
Learning from Synthetic Humans
Varol, G., Romero, J., Martin, X., Mahmood, N., Black, M. J., Laptev, I., & Schmid, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.492 · Cited by 908
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.139 · Cited by 884
Self-citationClustered Pose and Nonlinear Appearance Models for Human Pose Estimation
Johnson, S., & Everingham, M. (2010) · Procedings of the British Machine Vision Conference 2010 · doi:10.5244/c.24.12 · Cited by 819
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs
Tatarchenko, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.230 · Cited by 742
Kokkinos, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.579 · Cited by 657
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach
Zhou, X., Huang, Q., Sun, X., Xue, X., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.51 · Cited by 599
Self-citationStatistical methods for tomographic image reconstruction
Geman, S., & McClure, D. E. (1987) · Medical Entomology and Zoology · Cited by 579
3D Human Pose Estimation = 2D Pose Estimation + Matching
Chen, C.-H., & Ramanan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.610 · Cited by 566
Compositional Human Pose Regression
Sun, X., Shang, J., Liang, S., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.284 · Cited by 542
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.500 · Cited by 541
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image
Tome, D., Russell, C., & Agapito, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.603 · Cited by 504
Flowing ConvNets for Human Pose Estimation in Videos
Pfister, T., Charles, J., & Zisserman, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.222 · Cited by 500
Model-based vision: a program to see a walking person
Hogg, D. (1983) · Image and Vision Computing · doi:10.1016/0262-8856(83)90003-3 · Cited by 467
3D Human Pose Estimation from Monocular Images with Deep Convolutional Neural Network
Li, S., & Chan, A. B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16808-1_23 · Cited by 443
OpenDR: An Approximate Differentiable Renderer
Loper, M. M., & Black, M. J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_11 · Cited by 441
Pose-conditioned joint angle limits for 3D human pose reconstruction
Akhter, I., & Black, M. J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298751 · Cited by 418
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
Self-citationLearning effective human pose estimation from inaccurate annotation
Johnson, S., & Everingham, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995318 · Cited by 411
3D Human Pose Estimation from a Single Image via Distance Matrix Regression
Moreno-Noguer, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.170 · Cited by 397
Estimating human shape and pose from a single image
Guan, P., Weiss, A., Balan, A. O., & Black, M. J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459300 · Cited by 356
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.401 · Cited by 343
Detailed Human Shape and Pose from Images
Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., & Haussecker, H. W. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383340 · Cited by 308
Home 3D body scans from noisy image and range data
Weiss, A., Hirshberg, D., & Black, M. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126465 · Cited by 306
Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs
von Marcard, T., Rosenhahn, B., Black, M. J., & Pons-Moll, G. (2017) · Computer Graphics Forum · doi:10.1111/cgf.13131 · Cited by 304
Reconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
LCR-Net: Localization-Classification-Regression for Human Pose
Rogez, G., Weinzaepfel, P., & Schmid, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.134 · Cited by 296
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.153 · Cited by 269
Robinette, K. M., Blackwell, S., Daanen, H., Boehmer, M., & Fleming, S. (2002) · doi:10.21236/ada406704 · Cited by 260
Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation
Tekin, B., Marquez-Neila, P., Salzmann, M., & Fua, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.425 · Cited by 242
Structured Prediction of 3D Human Pose with Deep Neural Networks
Tekin, B., Katircioglu, I., Salzmann, M., Lepetit, V., & Fua, P. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.130 · Cited by 236
MonoPerfCap: Human Performance Capture From Monocular Video
Xu, W., Chatterjee, A., Zollhöfer, M., Rhodin, H., Mehta, D., Seidel, H.-P., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3181973 · Cited by 215
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
Rogez, G., & Schmid, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02046 · Cited by 210
Towards Accurate Marker-Less Human Shape and Pose Estimation over Time
Huang, Y., Bogo, F., Lassner, C., Kanazawa, A., Gehler, P. V., Romero, J., Akhter, I., & Black, M. J. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00055 · Cited by 207
Combined discriminative and generative articulated pose and non-rigid shape estimation
Sigal, L., Bălan, A. O., & Black, M. J. (2007) · Cited by 186
Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach
Zhou, X., Zhu, M., Leonardos, S., & Daniilidis, K. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2605097 · Cited by 145
Self-citationDeep Multitask Architecture for Integrated 2D and 3D Human Sensing
Popa, A.-I., Zanfir, M., & Sminchisescu, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.501 · Cited by 141
Indirect deep structured learning for 3D human body shape and pose prediction
Tan, V., Budvytis, I., & Cipolla, R. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.15 · Cited by 106
Human Shape from Silhouettes Using Generative HKS Descriptors and Cross-Modal Neural Networks
Dibra, E., Jain, H., Öztireli, C., Ziegler, R., & Groß, M. H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.584 · Cited by 100
General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues
Rhodin, H., Robertini, N., Casas, D., Richardt, C., Seidel, H., & Theobalt, C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_31 · Cited by 98
Generating Multiple Diverse Hypotheses for Human 3D Pose Consistent with 2D Joint Detections
Jahangiri, E., & Yuille, A. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.100 · Cited by 98
HS-Nets: Estimating Human Body Shape from Silhouettes with Convolutional Neural Networks
Dibra, E., Jain, H., Öztireli, C., Ziegler, R., & Groß, M. H. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.19 · Cited by 89
Optical Flow-Based 3D Human Motion Estimation from Monocular Video
Alldieck, T., Kassubeck, M., Wandt, B., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66709-6_28 · Cited by 51
MonoPerfCap: Human Performance Capture from Monocular VideoPreprint
Xu, W., Chatterjee, A., Zollhöfer, M., Rhodin, H., Mehta, D., Seidel, H., & Theobalt, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02136 · Cited by 44
Shape from Selfies: Human Body Shape Estimation Using CCA Regression Forests
Dibra, E., Öztireli, C., Ziegler, R., et al. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_6 · Cited by 39
Cited by
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationShi, F., Wang, J., Shi, J., Wu, Z., Wang, Q., Tang, Z., He, K., Shi, Y., & Shen, D. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.2987975 · Cited by 1,404
End-to-End Human Pose and Mesh Reconstruction with Transformers
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00199 · Cited by 740
Learning Joint Reconstruction of Hands and Manipulated Objects
Hasson, Y., Varol, G., Tzionas, D., Kalevatykh, I., Black, M. J., Laptev, I., & Schmid, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01208 · Cited by 513
Monocular human pose estimation: A survey of deep learning-based methods
Chen, Y., Tian, Y., & He, M. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.102897 · Cited by 441
Metadata
- Authors
- Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 459-468
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2798637590
- Cited by
- 607
- References
- 56
- FWCI
- 22.62
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
