The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Abstract
This paper addresses the challenge of 3D human pose estimation from a single color image. Despite the general success of the end-to-end learning paradigm, top performing approaches employ a two-step solution consisting of a Convolutional Network (ConvNet) for 2D joint localization and a subsequent optimization step to recover 3D pose. In this paper, we identify the representation of 3D pose as a critical issue with current ConvNet approaches and make two important contributions towards validating the value of end-to-end learning for this task. First, we propose a fine discretization of the 3D space around the subject and train a ConvNet to predict per voxel likelihoods for each joint. This creates a natural representation for 3D pose and greatly improves performance over the direct regression of joint coordinates. Second, to further improve upon initial estimates, we employ a coarse-to-fine prediction scheme. This step addresses the large dimensionality increase and enables iterative refinement and repeated processing of the image features. The proposed approach outperforms all state-of-the-art methods on standard benchmarks achieving a relative error reduction greater than 30% on average. Additionally, we investigate using our volumetric representation in a related architecture which is suboptimal compared to our end-to-end approach, but is of practical interest, since it enables training when no image with corresponding 3D groundtruth is available, and allows us to present compelling results for in-the-wild images.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
Sigal, L., Balan, A. O., & Black, M. J. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0273-6 · Cited by 1,164
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Zhuowen Tu, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 993
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 915
Deeplysupervised netsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015) · Cited by 849
Human Pose Estimation with Iterative Error Feedback
Carreira, J., Agrawal, P., Fragkiadaki, K., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.512 · Cited by 805
Recovering 3D human pose from monocular images
Agarwal, A., & Triggs, B. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.21 · Cited by 727
Monocular 3D pose estimation and tracking by detection
Andriluka, M., Roth, S., & Schiele, B. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540156 · Cited by 508
Flowing ConvNets for Human Pose Estimation in Videos
Pfister, T., Charles, J., & Zisserman, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.222 · Cited by 500
3D Human Pose Estimation from Monocular Images with Deep Convolutional Neural Network
Li, S., & Chan, A. B. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16808-1_23 · Cited by 443
3D People Tracking with Gaussian Process Dynamical Models
Urtasun, R., Fleet, D. J., & Fua, P. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.15 · Cited by 439
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
Self-citation3D Human pose estimation: A review of the literature and analysis of covariates
Sarafianos, N., Boteanu, B., Ionescu, B., & Kakadiaris, I. A. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.09.002 · Cited by 323
Reconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
3D Pictorial Structures for Multiple Human Pose Estimation
Belagiannis, V., Amin, S., Andriluka, M., Schiele, B., Navab, N., & Ilic, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.216 · Cited by 258
Deep Kinematic Pose Regression
Zhou, X., Sun, X., Zhang, W., Liang, S., & Wei, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_17 · Cited by 253
Self-citationTwin Gaussian Processes for Structured Prediction
Bo, L., & Sminchisescu, C. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-008-0204-y · Cited by 250
Single Image 3D Interpreter Network
Tian, Y., Wu, J., Xue, T., Lim, J. J., Tenenbaum, J. B., Torralba, A. B., & Freeman, W. T. (2016) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 241
Synthesizing Training Images for Boosting Human 3D Pose Estimation
Chen, W., Wang, H., Li, Y., Su, H., Wang, Z., Tu, C., Lischinski, D., Cohen-Or, D., & Chen, B. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.58 · Cited by 238
Structured Prediction of 3D Human Pose with Deep Neural Networks
Tekin, B., Katircioglu, I., Salzmann, M., Lepetit, V., & Fua, P. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.130 · Cited by 236
Optimization and Filtering for Human Motion Capture: A Multi-Layer Framework
Gall, J., Rosenhahn, B., Brox, T., & Seidel, H.-P. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-008-0173-1 · Cited by 229
Direct Prediction of 3D Body Poses from Motion Compensated Sequences
Tekin, B., Rozantsev, A., Lepetit, V., & Fua, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.113 · Cited by 215
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
Rogez, G., & Schmid, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02046 · Cited by 210
Determination of 3D human body postures from a single view
Lee, H.-J., & Chen, Z. (1985) · Computer Vision, Graphics, and Image Processing · doi:10.1016/0734-189x(85)90094-5 · Cited by 195
Harvesting Multiple Views for Marker-Less 3D Human Pose Annotations
Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.138 · Cited by 192
Self-citationA Dual-Source Approach for 3D Pose Estimation from a Single Image
Yasin, H., Iqbal, U., Kruger, B., Weber, A., & Gall, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.535 · Cited by 189
Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimation
Li, S., Zhang, W., & Chan, A. B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.326 · Cited by 181
3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information
Park, S., Hwang, J., & Kwak, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_15 · Cited by 160
3D Pictorial Structures for Multiple View Articulated Pose Estimation
Burenius, M., Sullivan, J., & Carlsson, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.464 · Cited by 158
Loose-limbed People: Estimating 3D Human Pose and Motion Using Non-parametric Belief Propagation
Sigal, L., Isard, M., Haussecker, H. W., & Black, M. J. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0493-4 · Cited by 157
Efficient ConvNet-based marker-less motion capture in general scenes with a low number of cameras
Elhayek, A., de Aguiar, E., Jain, A., Tompson, J., Pishchulin, L., Andriluka, M., Bregler, C., Schiele, B., & Theobalt, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299005 · Cited by 149
A Joint Model for 2D and 3D Pose Estimation from a Single Image
Simo‐Serra, E., Quattoni, A., Torras, C., et al. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.466 · Cited by 140
Single image 3D human pose estimation from noisy observations
Simo‐Serra, E., Arnau Ramisa, A., Alenyà, G., Carme Torras, A., & Moreno-Noguer, F. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247988 · Cited by 133
3D shape estimation from 2D landmarks: A convex relaxation approach
Zhou, X., Leonardos, S., Hu, X., & Daniilidis, K. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299074 · Cited by 131
Self-citationMarker-Less 3D Human Motion Capture with Monocular Image Sequence and Height-Maps
Du, Y., Wong, Y., Liu, Y., Han, F., Gui, Y., Wang, Z., Kankanhalli, M., & Geng, W. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_2 · Cited by 108
Multi-view Body Part Recognition with Random Forests
Kazemi, V., Burénius, M., Azizpour, H., et al. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.48 · Cited by 86
Depth Sweep Regression Forests for Estimating 3D Human Pose from Images
Kostrikov, I., & Gall, J. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.80 · Cited by 82
Bayesian Image Based 3D Pose Estimation
Sanzari, M., Ntouskos, V., & Pirri, F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_34 · Cited by 62
Learning Camera Viewpoint Using CNN to Improve 3D Body Pose Estimation
Fathollahi, M., Kasturi, R., & Sarkar, S. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.75 · Cited by 50
Cited by
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
A Simple Yet Effective Baseline for 3d Human Pose Estimation
Martinez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,462
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training
Pavllo, D., Feichtenhofer, C., Grangier, D., & Auli, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00794 · Cited by 1,297
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00064 · Cited by 1,142
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
Metadata
- Authors
- Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 1263-1272
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2554247908
- Cited by
- 884
- References
- 44
- FWCI
- 27.42
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.