The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
3D Hand Shape and Pose Estimation From a Single RGB Image
Open accessAbstract
This work addresses a novel and challenging problem of estimating the full 3D hand shape and pose from a single RGB image. Most current methods in 3D hand analysis from monocular RGB images only focus on estimating the 3D locations of hand keypoints, which cannot fully express the 3D shape of hand. In contrast, we propose a Graph Convolutional Neural Network (Graph CNN) based method to reconstruct a full 3D mesh of hand surface that contains richer information of both 3D hand shape and pose. To train networks with full supervision, we create a large-scale synthetic dataset containing both ground truth 3D meshes and 3D poses. When fine-tuning the networks on real-world datasets without 3D ground truth, we propose a weakly-supervised approach by leveraging the depth map as a weak supervision in training. Through extensive evaluations on our proposed new datasets and two public datasets, we show that our proposed method can produce accurate and reasonable 3D hand mesh, and can achieve superior 3D hand pose estimation accuracy when compared with state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410
Advances In Deep Learning On Graphs (Gsp'18 Workshop)
Defferrard, M. (2016) · Figshare · doi:10.5281/zenodo.1286817 · Cited by 4,613
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., & Jiang, Y.-G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_4 · Cited by 1,306
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., & Xiao, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03365 · Cited by 1,245
Hand Keypoint Detection in Single Images Using Multiview Bootstrapping
Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.494 · Cited by 1,119
Romero (2017) · doi:10.1145/3130800.3130883 · Cited by 1,056
Weighted Graph Cuts without Eigenvectors A Multilevel Approach
Dhillon, I. S., Guan, Y., & Kulis, B. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1115 · Cited by 1,003
Kato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954
Efficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874
Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks
Tompson, J., Stein, M., Lecun, Y., & Perlin, K. (2014) · ACM Transactions on Graphics · doi:10.1145/2629500 · Cited by 801
Learning to Estimate 3D Hand Pose from Single RGB Images
Zimmermann, C., & Brox, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.525 · Cited by 761
Li, Z., & Hoiem, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_37 · Cited by 659
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00055 · Cited by 607
Generating 3D Faces Using Convolutional Mesh Autoencoders
Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_43 · Cited by 560
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.500 · Cited by 541
GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB
Mueller, F., Bernard, F., Sotnychenko, O., Mehta, D., Sridhar, S., Casas, D., & Theobalt, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00013 · Cited by 521
Visual tracking of high DOF articulated structures: An application to human hand tracking
Rehg, J. M., & Kanade, T. (1994) · Lecture Notes in Computer Science · doi:10.1007/bfb0028333 · Cited by 453
Model-based hand tracking using a hierarchical Bayesian filter
Stenger, B., Thayananthan, A., Torr, P. H. S., & Cipolla, R. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.189 · Cited by 415
BodyNet: Volumetric Inference of 3D Human Body Shapes
Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., & Schmid, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_2 · Cited by 385
3D Hand Shape and Pose From Images in the Wild
Boukhayma, A., de, R., & Torr, P. H. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01110 · Cited by 326
FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis
Verma, N., Boyer, E., & Verbeek, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00275 · Cited by 324
Hand Pose Estimation via Latent 2.5D Heatmap Regression
Iqbal, U., Molchanov, P., Breuel, T., Gall, J., & Kautz, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_8 · Cited by 316
Hand PointNet: 3D Hand Pose Estimation Using Point Sets
Ge, L., Cai, Y., Weng, J., & Yuan, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00878 · Cited by 311
Self-citationGe, L., Liang, H., Yuan, J., & Thalmann, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.602 · Cited by 291
Self-citationWeakly-Supervised 3D Hand Pose Estimation from Monocular RGB Images
Cai, Y., Ge, L., Cai, J., & Yuan, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_41 · Cited by 291
Self-citationRobust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs
Ge, L., Liang, H., Yuan, J., & Thalmann, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.391 · Cited by 289
Self-citationCross-Modal Deep Variational Hand Pose Estimation
Spurr, A., Song, J., Park, S., & Hilliges, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00017 · Cited by 281
Model-Based 3D Hand Pose Estimation from Monocular Video
de La Gorce, M., Fleet, D. J., & Paragios, N. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.33 · Cited by 268
Self-supervised Learning of Motion CapturePreprint
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01337 · Cited by 262
Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data
Sridhar, S., Oulasvirta, A., & Theobalt, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.305 · Cited by 247
Real-Time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
Mueller, F., Mehta, D., Sotnychenko, O., Sridhar, S., Casas, D., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.131 · Cited by 242
Real-Time Joint Tracking of a Hand Manipulating an Object from RGB-D Input
Sridhar, S., Mueller, F., Zollhöfer, M., Casas, D., Oulasvirta, A., & Theobalt, C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_19 · Cited by 240
Hand modeling, analysis and recognition
Wu, Y., & Huang, T. S. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.924889 · Cited by 222
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Yuan, S., Garcia-Hernando, G., Stenger, B., Moon, G., Chang, J. Y., Lee, K. M., Molchanov, P., Kautz, J., Honari, S., Ge, L., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00279 · Cited by 217
Self-citationUsing a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild
Panteleris, P., Oikonomidis, I., & Argyros, A. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00054 · Cited by 201
Point-to-Point Regression PointNet for 3D Hand Pose Estimation
Ge, L., Ren, Z., & Yuan, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_29 · Cited by 167
Self-citationLearning an efficient model of hand shape variation from depth images
Khamis, S., Taylor, J., Shotton, J., Keskin, C., Izadi, S., & Fitzgibbon, A. W. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298869 · Cited by 115
3D Hand Pose Tracking and Estimation Using Stereo MatchingPreprint
Zhang, J., Jiao, J., Chen, M., Qu, L., Xu, X., & Yang, Q. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.07214 · Cited by 108
Indirect deep structured learning for 3D human body shape and pose prediction
Tan, V., Budvytis, I., & Cipolla, R. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.15 · Cited by 106
Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks
Ge, L., Liang, H., Yuan, J., & Thalmann, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2827052 · Cited by 98
Self-citationOnline generative model personalization for hand tracking
Tkach, A., Tagliasacchi, A., Remelli, E., Pauly, M. V., & Fitzgibbon, A. W. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130830 · Cited by 89
DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
Malik, J. N., Elhayek, A., Nunnari, F., Varanasi, K., Tamaddon, K., Héloir, A., & Stricker, D. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00023 · Cited by 85
Fits Like a Glove: Rapid and Reliable Hand Shape Personalization
Tan, D. J., Cashman, T. J., Taylor, J., Fitzgibbon, A. W., Tarlow, D., Khamis, S., Izadi, S., & Shotton, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.605 · Cited by 82
User-Specific Hand Modeling from Monocular Depth Sequences
Taylor, J., Stebbing, R. V., Ramakrishna, V., Keskin, C., Shotton, J., Izadi, S., Hertzmann, A., & Fitzgibbon, A. W. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.88 · Cited by 79
Robust 3D Hand Pose Estimation From Single Depth Images Using Multi-View CNNs
Ge, L., Liang, H., Yuan, J., & Thalmann, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2834824 · Cited by 77
Self-citationDomain Transfer for 3D Pose Estimation from Color Images without Manual AnnotationsPreprint
Rad, M., Oberweger, M., & Lepetit, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03707 · Cited by 52
Remelli, E., Tkach, A., Tagliasacchi, A., & Pauly, M. V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.277 · Cited by 40
Generating 3D faces using Convolutional Mesh AutoencodersPreprint
Ranjan, A., Bolkart, T., Sanyal, S., & Black, M. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.10267 · Cited by 37
Hough Forest With Optimized Leaves for Global Hand Pose Estimation With Arbitrary Postures
Liang, H., Yuan, J., Lee, J., Ge, L., & Thalmann, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2779800 · Cited by 30
Self-citationModel-based 3D Hand Tracking with on-line Shape Adaptation
Makris, A., & Argyros, A. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.77 · Cited by 26
Cited by
Hand Gesture Recognition Based on Computer Vision: A Review of Techniques
Oudah, M., Al-Naji, A., & Chahl, J. (2020) · Journal of Imaging · doi:10.3390/jimaging6080073 · Cited by 635
MediaPipe Hands: On-device Real-time Hand Tracking
Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.-L., & Grundmann, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10214 · Cited by 557
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks
Cai, Y., Ge, L., Liu, J., Cai, J., Cham, T.-J., Yuan, J., & Thalmann, N. M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00236 · Cited by 519
Self-citationFreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images
Zimmermann, C., Ceylan, D., Yang, J., Russell, B., Argus, M. J., & Brox, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00090 · Cited by 445
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose
Choi, H., Moon, G., & Lee, K. M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_45 · Cited by 392
Metadata
- Authors
- Ge, L., Ren, Z., Li, Y., Xue, Z., Wang, Y., Cai, J., & Yuan, J.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 10825-10834
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964093990
- Cited by
- 458
- References
- 57
- FWCI
- 54.10
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.