The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation
Open accessAbstract
Model-based 3D pose and shape estimation methods reconstruct a full 3D mesh for the human body by estimating several parameters. However, learning the abstract parameters is a highly non-linear process and suffers from image-model misalignment, leading to mediocre model performance. In contrast, 3D keypoint estimation methods combine deep CNN network with the volumetric representation to achieve pixel-level localization accuracy but may predict unrealistic body structure. In this paper, we address the above issues by bridging the gap between body mesh estimation and 3D keypoint estimation. We propose a novel hybrid inverse kinematics solution (HybrIK). HybrIK directly transforms accurate 3D joints to relative body-part rotations for 3D body mesh reconstruction, via the twistand-swing decomposition. The swing rotation is analytically solved with 3D joints, and the twist rotation is derived from the visual cues through the neural network. We show that HybrIK preserves both the accuracy of 3D pose and the realistic body structure of the parametric human model, leading to a pixel-aligned 3D body mesh and a more accurate 3D pose than the pure 3D keypoint estimation methods. Without bells and whistles, the proposed method surpasses the state-of-the-art methods by a large margin on various 3D human pose and shape benchmarks. As an illustrative example, HybrIK outperforms all the previous methods by 13.2 mm MPJPE and 21.9 mm PVE on 3DPW dataset. Our code is available at https://github.com/Jeff-sjtu/HybrIK.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 1×
“Two-stage approaches highly rely on the accurate 2D pose estimators, which have achieved impressive performance by the combination of powerful backbone network [58, 15, 42, 44, 45] and the 2D heatmap.”
MethodLinear and nonlinear programming.
(1986) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(86)90092-3 · Cited by 4,805
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1×
“6M [16] can be obtained by a linear combination of the mesh vertices through a linear regressor W , i.”
BackgroundEnd-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
A Simple Yet Effective Baseline for 3d Human Pose Estimation
Martinez, J., Hossain, R., Romero, J., & Little, J. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.288 · Cited by 1,462
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision
Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., & Theobalt, C. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00064 · Cited by 1,142
Review of pseudoinverse control for use with kinematically redundant manipulators
Klein, C. A., & Huang, C.-H. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313123 · Cited by 1,118
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera
von Marcard, T., Henschel, R., Black, M. J., Rosenhahn, B., & Pons-Moll, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_37 · Cited by 1,058
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
Kolotouros, N., Pavlakos, G., Black, M., & Daniilidis, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00234 · Cited by 1,039
VIBE: Video Inference for Human Body Pose and Shape Estimation
Kocabas, M., Athanasiou, N., & Black, M. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00530 · Cited by 1,031
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose
Pavlakos, G., Zhou, X., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.139 · Cited by 884
Integral Human Pose Regression
Sun, X., Xiao, B., Wei, F., Liang, S., & Wei, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_33 · Cited by 874
Wampler, C. (1986) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1986.289285 · Cited by 850 · cited 1×
“Numerical solutions [6, 70, 12, 19, 68, 9] are simple ways to implement the IK process, but they suffer from time-consuming iterative optimization.”
MethodSMPL: A Skinned Multi-Person Linear Model
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596800 · Cited by 768
Linear and nonlinear programming
Berck, P., & Sydsæter, K. (1993) · Economists’ Mathematical Manual · doi:10.1007/978-3-662-11597-8_14 · Cited by 667
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00055 · Cited by 607
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach
Zhou, X., Huang, Q., Sun, X., Xue, X., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.51 · Cited by 599
Real-Time Inverse Kinematics Techniques for Anthropomorphic Limbs
Tolani, D., Goswami, A., & Badler, N. I. (2000) · Graphical Models · doi:10.1006/gmod.2000.0528 · Cited by 591
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Kolotouros, N., Pavlakos, G., & Daniilidis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00463 · Cited by 548
Compositional Human Pose Regression
Sun, X., Shang, J., Liang, S., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.284 · Cited by 542
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.500 · Cited by 541
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498
FABRIK: A fast, iterative solver for the Inverse Kinematics problem
Aristidou, A., & Lasenby, J. (2011) · Graphical Models · doi:10.1016/j.gmod.2011.05.003 · Cited by 429
Pose-conditioned joint angle limits for 3D human pose reconstruction
Akhter, I., & Black, M. J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298751 · Cited by 418
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video
Zhou, X., Zhu, M., Leonardos, S., Derpanis, K. G., & Daniilidis, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.537 · Cited by 413
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM SIGGRAPH 2005 Papers · doi:10.1145/1186822.1073207 · Cited by 410
3D skeleton-based human action classification: A survey
Lo Presti, L., & La Cascia, M. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.11.019 · Cited by 397
3D Human Pose Estimation from a Single Image via Distance Matrix Regression
Moreno-Noguer, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.170 · Cited by 397
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB
Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Sridhar, S., Pons-Moll, G., & Theobalt, C. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00024 · Cited by 391
BodyNet: Volumetric Inference of 3D Human Body Shapes
Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., & Schmid, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_2 · Cited by 385
Moon, G., & Lee, K. M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_44 · Cited by 377
Estimating human shape and pose from a single image
Guan, P., Weiss, A., Balan, A. O., & Black, M. J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459300 · Cited by 356
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image
Moon, G., Chang, J. Y., & Lee, K. M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01023 · Cited by 344
Selectively Damped Least Squares for Inverse Kinematics
Buss, S. R., & Kim, J.-S. (2005) · Journal of Graphics Tools · doi:10.1080/2151237x.2005.10129202 · Cited by 338
Self-supervised Learning of Motion CapturePreprint
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01337 · Cited by 337
Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation
Fang, H.-S., Xu, Y., Wang, W., Liu, X., & Zhu, S.-C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12270 · Cited by 332
Computational modeling for the computer animation of legged figures
Girard, M., & Maciejewski, A. A. (1985) · ACM SIGGRAPH Computer Graphics · doi:10.1145/325165.325244 · Cited by 314
Reconstructing 3D Human Pose from 2D Image Landmarks
Ramakrishna, V., Kanade, T., & Sheikh, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_41 · Cited by 296
LCR-Net: Localization-Classification-Regression for Human Pose
Rogez, G., Weinzaepfel, P., & Schmid, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.134 · Cited by 296
Zanfir, A., Marinoiu, E., & Sminchisescu, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00229 · Cited by 292
Deep Kinematic Pose Regression
Zhou, X., Sun, X., Zhang, W., Liang, S., & Wei, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_17 · Cited by 253
Real-Time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
Mueller, F., Mehta, D., Sotnychenko, O., Sridhar, S., Casas, D., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.131 · Cited by 242
Neural Kinematic Networks for Unsupervised Motion Retargetting
Villegas, R., Yang, J., Ceylan, D., & Lee, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00901 · Cited by 240
HoloPose: Holistic 3D Human Reconstruction In-The-Wild
Guler, R. A., & Kokkinos, I. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01114 · Cited by 221
PaStaNet: Toward Human Activity Knowledge Engine
Li, Y., Xu, L., Liu, X., Huang, X., Xu, Y., Wang, S., Fang, H.-S., Ma, Z., Chen, M., & Lu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00046 · Cited by 198
Self-citationA Dual-Source Approach for 3D Pose Estimation from a Single Image
Yasin, H., Iqbal, U., Kruger, B., Weber, A., & Gall, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.535 · Cited by 189
Combined discriminative and generative articulated pose and non-rigid shape estimation
Sigal, L., Bălan, A. O., & Black, M. J. (2007) · Cited by 186
Detailed 2D-3D Joint Representation for Human-Object Interaction
Li, Y., Liu, X., Han, L., Wang, S., Liu, J., Li, J., & Lu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01018 · Cited by 181
Self-citationQuaterNet: A Quaternion-based Recurrent Model for Human Motion
Pavllo, D., Grangier, D., & Auli, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06485 · Cited by 174
3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information
Park, S., Hwang, J., & Kwak, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_15 · Cited by 160
Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach
Zhou, X., Zhu, M., Leonardos, S., & Daniilidis, K. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2605097 · Cited by 145
Tung, H.-Y. F., Harley, A. W., Seto, W., & Fragkiadaki, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.467 · Cited by 143
Learning features combination for human action recognition from skeleton sequences
Luvizon, D. C., Tabia, H., & Picard, D. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.02.001 · Cited by 117
On solving the inverse kinematics problem using neural networks
Csiszar, A., Eilers, J., & Verl, A. W. (2017) · International Conference on Mechatronics and Machine Vision in Practice (M2VIP) · doi:10.1109/m2vip.2017.8211457 · Cited by 89
HMOR: Hierarchical Multi-person Ordinal Relations for Monocular Multi-person 3D Pose Estimation
Wang, C., Li, J., Liu, W., Qian, C., & Lu, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_15 · Cited by 74
Self-citationBodyNet: Volumetric Inference of 3D Human Body ShapesPreprint
Varol, G., Ceylan, D., Russell, B. C., Yang, J., Yumer, E., Ivan Laptev, A., & Schmid, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04875 · Cited by 63
Bayesian Image Based 3D Pose Estimation
Sanzari, M., Ntouskos, V., & Pirri, F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_34 · Cited by 62
Deep RNN Framework for Visual Sequential Applications
Pang, B., Zha, K., Cao, H., Chen, S., & Lu, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00051 · Cited by 57
Self-citationIK-FA, a New Heuristic Inverse Kinematics Solver Using Firefly Algorithm
Rokbani, N., Casals, A., & Alimi, A. M. (2014) · Studies in computational intelligence · doi:10.1007/978-3-319-11017-2_15 · Cited by 56
Learning to Estimate Pose and Shape of Hand-Held Objects from RGB Images
Kokić, M., Kragić, D., & Bohg, J. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967961 · Cited by 40
Complex sequential understanding through the awareness of spatial and temporal concepts
Pang, B., Zha, K., Cao, H., Tang, J., Yu, M., & Lu, C. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0168-3 · Cited by 24
Self-citationSkeleton Transformer Networks: 3D Human Pose and Skinned Mesh from Single RGB Image
Yoshiyasu, Y., Sagawa, R., Ayusawa, K., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_30 · Cited by 20
Cited by
MotionBERT: A Unified Perspective on Learning Human Motion Representations
Zhu, W., Ma, X., Liu, Z., Liu, L., Wu, W., & Wang, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01385 · Cited by 348
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Zhang, H., Tian, Y., Zhou, X., Ouyang, W., Liu, Y., Wang, L., & Sun, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01125 · Cited by 341
CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation
Li, Z., Liu, J., Zhang, Z., Xu, S., & Yan, Y. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20065-6_34 · Cited by 268
Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model
Kim, J.-W., Choi, J.-Y., Ha, E.-J., & Choi, J.-H. (2023) · Applied Sciences · doi:10.3390/app13042700 · Cited by 256
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Grauman, K., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, T., Ashutosh, K., Baiyya, V., Bansal, S., Boote, B., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01834 · Cited by 182
Metadata
- Authors
- Li, J., Xu, C., Chen, Z., Bian, S., Yang, L., & Lu, C.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 3382-3392
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3167491448
- Cited by
- 402
- References
- 67
- FWCI
- 23.39
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.