The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
ICON: Implicit Clothed humans Obtained from Normals
Open accessAbstract
Current methods for learning realistic and animatable 3D clothed avatars need either posed 3D scans or 2D images with carefully controlled user poses. In contrast, our goal is to learn an avatar from only 2D images of people in unconstrained poses. Given a set of images, our method estimates a detailed 3D surface from each image and then combines these into an animatable avatar. Implicit functions are well suited to the first task, as they can capture details like hair and clothes. Current methods, however, are not robust to varied human poses and often produce 3D surfaces with broken or disembodied limbs, missing details, or non-human shapes. The problem is that these methods use global feature encoders that are sensitive to global pose. To address this, we propose ICON (“Tmplicit Clothed humans Obtained from Normals”), which, instead, uses local features. ICON has two main modules, both of which exploit the SMPL(-X) body model. First, ICON infers detailed clothed-human normals (front/back) conditioned on the SMPL(-X) normals. Second, a visibility-aware implicit surface regressor produces an iso-surface of a human occupancy field. Importantly, at inference time, a feedback loop alternates between refining the SMPL(-X) mesh using the inferred clothed normals and then refining the normals. Given multiple reconstructed frames of a subject in varied poses, we use a modified version of SCANimate to produce an animatable avatar from them. Evaluation on the AGORA and CAPE datasets shows that ICON outperforms the state of the art in reconstruction, even with heavily limited training data. Additionally, it is much more robust to out-of-distribution samples, e.g., in-the-wild poses/images and out-of-frame cropping. ICON takes a step towards robust 3D clothed human reconstruction from in-the-wild images. This enables avatar creation directly from video with personalized pose-dependent cloth deformation. Models and code are available for research at https://icon.is.tue.mpg.de.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Self-citationExpressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationLearning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
Learning to Reconstruct 3D Human Pose and Shape via Model-Fitting in the Loop
Kolotouros, N., Pavlakos, G., Black, M., & Daniilidis, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00234 · Cited by 1,039
Self-citationVIBE: Video Inference for Human Body Pose and Shape Estimation
Kocabas, M., Athanasiou, N., & Black, M. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00530 · Cited by 1,031
Self-citationPIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Saito, S., Simon, T., Saragih, J., & Joo, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00016 · Cited by 793
SMPL: A Skinned Multi-Person Linear Model
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596800 · Cited by 768
Self-citationMarching cubes: a high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1998) · Seminal graphics · doi:10.1145/280811.281026 · Cited by 673
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies
Joo, H., Simon, T., & Sheikh, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00868 · Cited by 509
Video Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
Multi-Garment Net: Learning to Dress 3D People From Images
Bhatnagar, B., Tiwari, G., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00552 · Cited by 439
PARE: Part Attention Regressor for 3D Human Body Estimation
Kocabas, M., Huang, C.-H. P., Hilliges, O., & Black, M. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01094 · Cited by 417
Self-citationAccelerating 3D deep learning with PyTorch3D
Johnson, J., Ravi, N., Reizenstein, J., Novotny, D., Tulsiani, S., Lassner, C., & Branson, S. (2020) · SIGGRAPH Asia 2020 Courses · doi:10.1145/3415263.3419160 · Cited by 413
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
Learning to Dress 3D People in Generative Clothing
Ma, Q., Yang, J., Ranjan, A., Pujades, S., Pons-Moll, G., Tang, S., & Black, M. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00650 · Cited by 364
Self-citationGHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models
Xu, H., Bazavan, E. G., Zanfir, A., Freeman, W. T., Sukthankar, R., & Sminchisescu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00622 · Cited by 347
ARCH: Animatable Reconstruction of Clothed Humans
Huang, Z., Xu, Y., Lassner, C., Li, H., & Tung, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00316 · Cited by 341
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop
Zhang, H., Tian, Y., Zhou, X., Ouyang, W., Liu, Y., Wang, L., & Sun, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01125 · Cited by 341
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
Alldieck, T., Pons-Moll, G., Theobalt, C., & Magnor, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00238 · Cited by 332
Learning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M., Bhatnagar, B. L., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00127 · Cited by 326
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction
Zheng, Z., Yu, T., Liu, Y., & Dai, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050505 · Cited by 320
Detailed, Accurate, Human Shape Estimation from Clothed 3D Scan Sequences
Zhang, C., Pujades, S., Black, M., & Pons-Moll, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.582 · Cited by 295
Self-citationSCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks
Saito, S., Yang, J., Ma, Q., & Black, M. J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00291 · Cited by 222
Self-citationDetailed Human Avatars from Monocular Video
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00022 · Cited by 206
Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human ReconstructionPreprint
He, T., Collomosse, J., Jin, H., & Soatto, S. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2006.08072 · Cited by 205
SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes
Chen, X., Zheng, Y., Black, M. J., Hilliges, O., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01139 · Cited by 201
Self-citationCombining Implicit Function Learning and Parametric Models for 3D Human Reconstruction
Bhatnagar, B. L., Sminchisescu, C., Theobalt, C., & Pons-Moll, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_19 · Cited by 200
AGORA: Avatars in Geography Optimized for Regression Analysis
Patel, P., Huang, C.-H. P., Tesch, J., Hoffmann, D. T., Tripathi, S., & Black, M. J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01326 · Cited by 197
Self-citationARCH++: Animation-Ready Clothed Human Reconstruction Revisited
He, T., Xu, Y., Saito, S., Soatto, S., & Tung, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01086 · Cited by 195
BCNet: Learning Body and Cloth Shape from a Single Image
Jiang, B., Zhang, J., Hong, Y., Luo, J., Liu, L., & Bao, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_2 · Cited by 180
Putting People in their Place: Monocular Regression of 3D People in Depth
Sun, Y., Liu, W., Bao, Q., Fu, Y., Mei, T., & Black, M. J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01289 · Cited by 166
Self-citationDetailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation
Zhu, H., Zuo, X.-X., Wang, S., Cao, X., & Yang, R.-G. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00462 · Cited by 165
Collaborative Regression of Expressive Bodies using Moderation
Feng, Y., Choutas, V., Bolkart, T., Tzionas, D., & Black, M. J. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00088 · Cited by 140
Self-citation360-Degree Textures of People in Clothing from a Single Image
Lazova, V., Insafutdinov, E., & Pons‐Moll, G. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00076 · Cited by 138
DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras
Zheng, Y., Shao, R., Zhang, Y., Yu, T., Zheng, Z., Dai, Q., & Liu, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00618 · Cited by 124
Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos
Jafarian, Y., & Park, H. S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01256 · Cited by 118
Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing
Tiwari, G., Sarafianos, N., Tung, T., & Pons‐Moll, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01150 · Cited by 105
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements
Ma, Q., Saito, S., Yang, J., Tang, S., & Black, M. J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01582 · Cited by 99
Self-citationThe Power of Points for Modeling Humans in Clothing
Ma, Q., Yang, J., Tang, S., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01079 · Cited by 95
Self-citationMonocular Real-Time Volumetric Performance Capture
Li, R., Xiu, Y., Saito, S., Huang, Z., Olszewski, K., & Li, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58592-1_4 · Cited by 93
Self-citationgDNA: Towards Generative Detailed Neural Avatars
Chen, X., Jiang, T., Song, J., Yang, J., Black, M. J., Geiger, A., & Hilliges, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01978 · Cited by 82
Self-citation3D Human Body Reconstruction from a Single Image via Volumetric Regression
Jackson, A. S., Manafas, C., & Tzimiropoulos, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_6 · Cited by 79
S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
Yang, Z., Wang, S., Manivasagam, S., Huang, Z., Ma, W.-C., Yan, X., Yumer, E., & Urtasun, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01308 · Cited by 79
Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction
Božič, A., Palafox, P. R., Zollhöfer, M., Thies, J., Dai, A., & Nießner, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00150 · Cited by 77
MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video
Xiang, D., Prada, F., Wu, C., & Hodgins, J. K. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00042 · Cited by 71
PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence
Dong, Z., Guo, C., Song, J., Chen, X., Geiger, A., & Hilliges, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01982 · Cited by 71
Human-Aware Object Placement for Visual Environment Reconstruction
Yi, H., Huang, C.-H. P., Tzionas, D., Kocabas, M., Hassan, M. S., Tang, S., Thies, J., & Black, M. J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00393 · Cited by 70
Self-citationFACSIMILE: Fast and Accurate Scans From an Image in Less Than a Second
Smith, D., Loper, M., Hu, X., Mavroidis, P., & Romero, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00543 · Cited by 62
MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
Wang, S., Mihajlović, M., Ma, Q., Geiger, A., & Tang, S. (2021) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000518561 · Cited by 58
Robust 3D Self-Portraits in Seconds
Li, Z., Yu, T., Pan, C., Zheng, Z., & Liu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00142 · Cited by 55
A Neural Network for Detailed Human Depth Estimation From a Single Image
Tang, S., Tan, F., Cheng, K., Li, Z., Zhu, S., & Tan, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00784 · Cited by 49
Metadata
- Authors
- Xiu, Y., Yang, J., Tzionas, D., & Black, M. J.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 13286-13296
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4226160668
- Cited by
- 318
- References
- 55
- FWCI
- 55.91
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.