The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
ARCH: Animatable Reconstruction of Clothed Humans
Abstract
In this paper, we propose ARCH (Animatable Reconstruction of Clothed Humans), a novel end-to-end framework for accurate reconstruction of animation-ready 3D clothed humans from a monocular image. Existing approaches to digitize 3D humans struggle to handle pose variations and recover details. Also, they do not produce models that are animation ready. In contrast, ARCH is a learned pose-aware model that produces detailed 3D rigged full-body human avatars from a single unconstrained RGB image. A Semantic Space and a Semantic Deformation Field are created using a parametric 3D body estimator. They allow the transformation of 2D/3D clothed humans into a canonical space, reducing ambiguities in geometry caused by pose variations and occlusions in training data. Detailed surface geometry and appearance are learned using an implicit function representation with spatial local features. Furthermore, we propose additional per-pixel supervision on the 3D reconstruction using opacity-aware differentiable rendering. Our experiments indicate that ARCH increases the fidelity of the reconstructed humans. We obtain more than 50% lower reconstruction errors for standard metrics compared to state-of-the-art methods on public datasets. We also show numerous qualitative examples of animated, high-quality reconstructed avatars unseen in the literature so far.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,647
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations
Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.124 · Cited by 1,803
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
Self-citationLearning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
Self-citationLearning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
Kolotouros, N., Pavlakos, G., Black, M., & Daniilidis, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00234 · Cited by 1,039
Kato, H., Ushiku, Y., & Harada, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00411 · Cited by 954
DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
Newcombe, R. A., Fox, D., & Seitz, S. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298631 · Cited by 951
Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., & McMillan, L. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344951 · Cited by 890
Accurate, Dense, and Robust Multi-View Stereopsis
Furukawa, Y., & Ponce, J. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383246 · Cited by 692
Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning
Liu, S., Chen, W., Li, T., & Li, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00780 · Cited by 641
Self-citationArticulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360696 · Cited by 610
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
Kolotouros, N., Pavlakos, G., & Daniilidis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00463 · Cited by 548
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.500 · Cited by 541
Self-citationNeural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498
Self-citationClothCap: seamless 4D clothing capture and retargeting
Pons-Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495
Video Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
OpenDR: An Approximate Differentiable Renderer
Loper, M. M., & Black, M. J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_11 · Cited by 441
Multi-Garment Net: Learning to Dress 3D People From Images
Bhatnagar, B., Tiwari, G., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00552 · Cited by 439
Differentiable Monte Carlo ray tracing through edge sampling
Li, T.-M., Aittala, M., Durand, F., & Lehtinen, J. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275109 · Cited by 428
BodyNet: Volumetric Inference of 3D Human Body Shapes
Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., & Schmid, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_2 · Cited by 385
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
Motion capture using joint skeleton tracking and surface estimation
Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., & Seidel, H.-P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206755 · Cited by 344
Self-supervised Learning of Motion CapturePreprint
Tung, H.-Y. F., Tung, H.-W., Yumer, E., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01337 · Cited by 337
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
Alldieck, T., Pons-Moll, G., Theobalt, C., & Magnor, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00238 · Cited by 332
Learning Pose Grammar to Encode Human Body Configuration for 3D Pose Estimation
Fang, H.-S., Xu, Y., Wang, W., Liu, X., & Zhu, S.-C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12270 · Cited by 332
Self-citationLearning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M., Bhatnagar, B. L., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00127 · Cited by 326
Detailed Human Shape and Pose from Images
Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., & Haussecker, H. W. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383340 · Cited by 308
Detailed, Accurate, Human Shape Estimation from Clothed 3D Scan Sequences
Zhang, C., Pujades, S., Black, M., & Pons-Moll, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.582 · Cited by 295
Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., Pons-Moll, G., & Liu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00761 · Cited by 255
Self-citationDynamic shape capture using multi-view photometric stereo
Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618520 · Cited by 238
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
Chen, W., Gao, J., Ling, H., Smith, E. J., Lehtinen, J., Jacobson, A., & Fidler, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01210 · Cited by 238
DeepWrinkles: Accurate and Realistic Clothing Modeling
Lähner, Z., Cremers, D., & Tung, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_41 · Cited by 225
Self-citationHoloPose: Holistic 3D Human Reconstruction In-The-Wild
Guler, R. A., & Kokkinos, I. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01114 · Cited by 221
SSD: Smooth Signed Distance Surface Reconstruction
Calakli, F., & Taubin, G. (2011) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2011.02058.x · Cited by 220
Detailed Human Avatars from Monocular Video
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00022 · Cited by 206
SiCloPe: Silhouette-Based Clothed People
Natsume, R., Saito, S., Huang, Z., Chen, W., Ma, C., Li, H., & Morishima, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00461 · Cited by 199
Self-citationLearning to Infer Implicit Surfaces without 3D SupervisionPreprint
Liu, S., Saito, S., Chen, W., & Li, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00767 · Cited by 186
Self-citationDenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare
Xu, Y., Zhu, S.-C., & Tung, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00785 · Cited by 182
Self-citationMulti-view People Tracking via Hierarchical Trajectory Composition
Xu, Y., Liu, X., Liu, Y., & Zhu, S.-C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.461 · Cited by 168
Self-citation360-Degree Textures of People in Clothing from a Single Image
Lazova, V., Insafutdinov, E., & Pons‐Moll, G. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00076 · Cited by 160
Dynamic shape capture using multi-view photometric stereo
Vlasic, D., Peers, P., Baran, I., Debevec, P. E., Popović, J., Rusinkiewicz, S. M., & Matusik, W. (2009) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1661412.1618520 · Cited by 120
HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs
Zheng, Z., Yu, T., Li, H., Guo, K., Dai, Q., Fang, L., & Liu, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_24 · Cited by 99
Self-citationEstimation of Human Body Shape in Motion with Wide Clothing
Yang, J., Franco, J.-S., Hétroy, F., & Wuhrer, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_27 · Cited by 85
3D Human Body Reconstruction from a Single Image via Volumetric Regression
Jackson, A. S., Manafas, C., & Tzimiropoulos, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_6 · Cited by 79
Tung, T., Nobuhara, S., & Matsuyama, T. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459384 · Cited by 54
Self-citationScene-Centric Joint Parsing of Cross-View Videos
Qi, H., Xu, Y., Yuan, T., Wu, T., & Zhu, S. C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12256 · Cited by 14
Self-citation
Cited by
Neural Fields in Visual Computing and Beyond
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., sitzmann, V., & Sridhar, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14505 · Cited by 484
HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video
Weng, C.-Y., Curless, B., Srinivasan, P. P., Barron, J. T., & Kemelmacher-Shlizerman, I. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01573 · Cited by 452
Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies
Peng, S., Dong, J., Wang, Q., Zhang, S., Shuai, Q., Zhou, X., & Bao, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01405 · Cited by 387
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction
Zheng, Z., Yu, T., Liu, Y., & Dai, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050505 · Cited by 320
ICON: Implicit Clothed humans Obtained from Normals
Xiu, Y., Yang, J., Tzionas, D., & Black, M. J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01294 · Cited by 318
Metadata
- Authors
- Huang, Z., Xu, Y., Lassner, C., Li, H., & Tung, T.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 3090-3099
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035492592
- Cited by
- 341
- References
- 53
- FWCI
- 52.03
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.