The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Open accessAbstract
3D models provide a common ground for different representations of human bodies. In turn, robust 2D estimation has proven to be a powerful tool to obtain 3D fits in-the-wild. However, depending on the level of detail, it can be hard to impossible to acquire labeled data for training 2D estimators on large scale. We propose a hybrid approach to this problem: with an extended version of the recently introduced SMPLify method, we obtain high quality 3D body model fits for multiple human pose datasets. Human annotators solely sort good and bad fits. This procedure leads to an initial dataset, UP-3D, with rich annotations. With a comprehensive set of experiments, we show how this data can be used to train discriminative models that produce results with an unprecedented level of detail: our models predict 31 segments and 91 landmark locations on the body. Using the 91 landmark pose estimator, we present state-of-the art results for 3D human pose and shape estimation using an order of magnitude less training data and without assumptions about gender or pose in the fitting procedure. We show that UP-3D can be enhanced with these improved fits to grow in quantity and quality, which makes the system deployable on large scale. The data, code and models are available for research purposes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Self-citationHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Pedestrian Detection: An Evaluation of the State of the Art
Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.155 · Cited by 3,190
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
2D Human Pose Estimation: New Benchmark and State of the Art Analysis
Andriluka, M., Pishchulin, L., Gehler, P., & Schiele, B. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.471 · Cited by 2,688
Self-citationRepresentation and recognition of the spatial organization of three-dimensional shapes
Marr, D., & Nishihara, H. K. (1978) · Proceedings of the Royal Society of London. Series B. Biological Sciences · doi:10.1098/rspb.1978.0020 · Cited by 2,158
Cited by
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Self-citationExpressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationPIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera
von Marcard, T., Henschel, R., Black, M. J., Rosenhahn, B., & Pons-Moll, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_37 · Cited by 1,058
Self-citationLearning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
Kolotouros, N., Pavlakos, G., Black, M., & Daniilidis, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00234 · Cited by 1,039
Self-citationVIBE: Video Inference for Human Body Pose and Shape Estimation
Kocabas, M., Athanasiou, N., & Black, M. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00530 · Cited by 1,031
Self-citationPIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Saito, S., Simon, T., Saragih, J., & Joo, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00016 · Cited by 793
End-to-End Human Pose and Mesh Reconstruction with Transformers
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00199 · Cited by 740
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00055 · Cited by 607
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Kolotouros, N., Pavlakos, G., & Daniilidis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00463 · Cited by 548
Learning 3D Human Dynamics From Video
Kanazawa, A., Zhang, J. Y., Felsen, P., & Malik, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00576 · Cited by 531
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498
Self-citationClothCap: seamless 4D clothing capture and retargeting
Pons-Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495
Self-citationVideo Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
3D Hand Shape and Pose Estimation From a Single RGB Image
Ge, L., Ren, Z., Li, Y., Xue, Z., Wang, Y., Cai, J., & Yuan, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01109 · Cited by 458
FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape From Single RGB Images
Zimmermann, C., Ceylan, D., Yang, J., Russell, B., Argus, M. J., & Brox, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00090 · Cited by 445
Monocular human pose estimation: A survey of deep learning-based methods
Chen, Y., Tian, Y., & He, M. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.102897 · Cited by 441
PARE: Part Attention Regressor for 3D Human Body Estimation
Kocabas, M., Huang, C.-H. P., Hilliges, O., & Black, M. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01094 · Cited by 417
Self-citationOrdinal Depth Supervision for 3D Human Pose Estimation
Pavlakos, G., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00763 · Cited by 411
Li, J., Xu, C., Chen, Z., Bian, S., Yang, L., & Lu, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00339 · Cited by 402
Deep 3D human pose estimation: A review
Wang, J., Tan, S., Zhen, X., Xu, S., Zheng, F., He, Z., & Shao, L. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103225 · Cited by 399
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose
Choi, H., Moon, G., & Lee, K. M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_45 · Cited by 392
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB
Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Sridhar, S., Pons-Moll, G., & Theobalt, C. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00024 · Cited by 391
3D Menagerie: Modeling the 3D Shape and Pose of Animals
Zuffi, S., Kanazawa, A., Jacobs, D. W., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.586 · Cited by 381
Self-citationBodyNet: Volumetric Inference of 3D Human Body Shapes
Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., & Schmid, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_2 · Cited by 385
Moon, G., & Lee, K. M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_44 · Cited by 377
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
Learning to Dress 3D People in Generative Clothing
Ma, Q., Yang, J., Ranjan, A., Pujades, S., Pons-Moll, G., Tang, S., & Black, M. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00650 · Cited by 364
Self-citationARCH: Animatable Reconstruction of Clothed Humans
Huang, Z., Xu, Y., Lassner, C., Li, H., & Tung, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00316 · Cited by 341
Self-citationPyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Zhang, H., Tian, Y., Zhou, X., Ouyang, W., Liu, Y., Wang, L., & Sun, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01125 · Cited by 341
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01270 · Cited by 337
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
Alldieck, T., Pons-Moll, G., Theobalt, C., & Magnor, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00238 · Cited by 332
Learning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M., Bhatnagar, B. L., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00127 · Cited by 326
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction
Zheng, Z., Yu, T., Liu, Y., & Dai, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050505 · Cited by 320
HuMoR: 3D Human Motion Model for Robust Pose Estimation
Rempe, D., Birdal, T., Hertzmann, A., Yang, J., Sridhar, S., & Guibas, L. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01129 · Cited by 304
Detailed, Accurate, Human Shape Estimation from Clothed 3D Scan Sequences
Zhang, C., Pujades, S., Black, M., & Pons-Moll, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.582 · Cited by 295
Self-citationDensePose: Dense Human Pose Estimation in the Wild
Guler, R. A., Neverova, N., & Kokkinos, I. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00762 · Cited by 290
LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images
Rogez, G., Weinzaepfel, P., & Schmid, C. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2892985 · Cited by 272
Humans in 4D: Reconstructing and Tracking Humans with Transformers
Goel, S., Pavlakos, G., Rajasegaran, J., Kanazawa, A., & Malik, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01358 · Cited by 272
Kimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211056674 · Cited by 271
Monocular, One-stage, Regression of Multiple 3D People
Sun, Y., Bao, Q., Liu, W., Fu, Y., Black, M. J., & Mei, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01099 · Cited by 254
Self-citationCategory-Level Articulated Object Pose Estimation
Li, X., Wang, H., Yi, L., Guibas, L. J., Abbott, A. L., & Song, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00376 · Cited by 196
Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture From Images “In the Wild”
Zuffi, S., Kanazawa, A., Berger-Wolf, T., & Black, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00546 · Cited by 139
“[15] build a dataset of human-rated high-quality 3D model fits.”
BackgroundSelf-citationSelf-supervised 3D Representation Learning of Dressed Humans from Social Media Videos
Jafarian, Y., & Park, H. S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3231558 · Cited by 12
Metadata
- Authors
- Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. V.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 4704-4713
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2573098616
- Cited by
- 541
- References
- 45
- FWCI
- 15.37
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.