The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Open accessAbstract
We introduce Pixel-aligned Implicit Function (PIFu), an implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object. Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images. Highly intricate shapes, such as hairstyles, clothing, as well as their variations and deformations can be digitized in a unified way. Compared to existing representations used for 3D deep learning, PIFu produces high-resolution surfaces including largely unseen regions such as the back of a person. In particular, it is memory efficient unlike the voxel representation, can handle arbitrary topology, and the resulting surface is spatially aligned with the input image. Furthermore, while previous techniques are designed to process either a single image or multiple views, PIFu extends naturally to arbitrary number of views. We demonstrate high-resolution and robust reconstructions on real world images from the DeepFashion dataset, which contains a variety of challenging clothing types. Our method achieves state-of-the-art performance on a public benchmark and outperforms the prior work for clothed human digitization from a single image.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,647
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,019
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
Self-citationDeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations
Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.124 · Cited by 1,803
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C. B., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,632
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
Self-citationSCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
Learning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
DensePose: Dense Human Pose Estimation in the Wild.
Güler, R. A., Neverova, N., & Kokkinos, I. (2018) · UCL Discovery (University College London) · Cited by 1,421
High-quality video view interpolation using a layered representation
Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., & Szeliski, R. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015766 · Cited by 1,385
Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation
Tompson, J., Jain, A., LeCun, Y., & Bregler, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.2984 · Cited by 915
Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., & McMillan, L. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344951 · Cited by 890
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360696 · Cited by 610
Learning to Estimate 3D Human Pose and Shape from a Single Color Image
Pavlakos, G., Zhu, L., Zhou, X., & Daniilidis, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00055 · Cited by 607
View Synthesis by Appearance Flow
Zhou, T., Tulsiani, S., Sun, W., Malik, J., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_18 · Cited by 579
Unite the People: Closing the Loop Between 3D and 2D Human Representations
Lassner, C., Romero, J., Kiefel, M., Bogo, F., Black, M. J., & Gehler, P. V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.500 · Cited by 541
Learning Category-Specific Mesh Reconstruction from Image Collections
Kanazawa, A., Tulsiani, S., Efros, A. A., & Malik, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_23 · Cited by 530
Self-citationMulti-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation
Omran, M., Lassner, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00062 · Cited by 498
ClothCap: seamless 4D clothing capture and retargeting
Pons-Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495
Video Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
Surface Capture for Performance-Based Animation
Starck, J., & Hilton, A. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.68 · Cited by 452
Silhouette and stereo fusion for 3D object modeling
Hernández Esteban, C., & Schmitt, F. (2004) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.03.016 · Cited by 435
AtlasNet: A Papier-Mâché Approach to Learning 3D Surface GenerationPreprint
Groueix, T., Fisher, M., Kim, V. G., Russell, B. C., & Aubry, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05384 · Cited by 409
BodyNet: Volumetric Inference of 3D Human Body Shapes
Varol, G., Ceylan, D., Russell, B., Yang, J., Yumer, E., Laptev, I., & Schmid, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_2 · Cited by 385
Embree: a kernel framework for efficient CPU ray tracing
Wald, I., Woop, S., Benthin, C., Johnson, G. S., & Ernst, M. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601199 · Cited by 378
SurfaceNet: An End-to-End 3D Neural Network for Multiview Stereopsis
Ji, M., Gall, J., Zheng, H., Liu, Y., & Fang, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.253 · Cited by 361
Estimating human shape and pose from a single image
Guan, P., Weiss, A., Balan, A. O., & Black, M. J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459300 · Cited by 356
Motion capture using joint skeleton tracking and surface estimation
Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., & Seidel, H.-P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206755 · Cited by 344
Learning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M., Bhatnagar, B. L., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00127 · Cited by 326
Detailed Human Shape and Pose from Images
Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., & Haussecker, H. W. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383340 · Cited by 308
Hierarchical Surface Prediction for 3D Object Reconstruction
Hane, C., Tulsiani, S., & Malik, J. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00054 · Cited by 307
Detailed, Accurate, Human Shape Estimation from Clothed 3D Scan Sequences
Zhang, C., Pujades, S., Black, M., & Pons-Moll, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.582 · Cited by 295
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
Park, E., Yang, J., Yumer, E., Ceylan, D., & Berg, A. C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.82 · Cited by 279
Dynamic shape capture using multi-view photometric stereo
Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618520 · Cited by 238
Learning a Multi-View Stereo Machine
Kar, A., Häne, C., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05375 · Cited by 209
Towards Accurate Marker-Less Human Shape and Pose Estimation over Time
Huang, Y., Bogo, F., Lassner, C., Kanazawa, A., Gehler, P. V., Romero, J., Akhter, I., & Black, M. J. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00055 · Cited by 207
Self-citationDetailed Human Avatars from Monocular Video
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00022 · Cited by 206
SiCloPe: Silhouette-Based Clothed People
Natsume, R., Saito, S., Huang, Z., Chen, W., Ma, C., Li, H., & Morishima, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00461 · Cited by 199
Self-citationNeverova, N., Alp Güler, R., & Kokkinos, I. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_8 · Cited by 196
Parametric reshaping of human bodies in images
Zhou, S., Fu, H., Liu, L., Cohen-Or, D., & Han, X. (2010) · ACM Transactions on Graphics · doi:10.1145/1778765.1778863 · Cited by 175
Fast winding numbers for soups and clouds
Barill, G., Dickson, N. G., Schmidt, R., Levin, D. I. W., & Jacobson, A. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201337 · Cited by 170
Photo Wake-Up: 3D Character Animation From a Single Photo
Weng, C.-Y., Curless, B., & Kemelmacher-Shlizerman, I. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00606 · Cited by 160
Learning Local Shape Descriptors from Part Correspondences with Multiview Convolutional Networks
Huang, H., Kalogerakis, E., Chaudhuri, S., Ceylan, D., Kim, V. G., & Yumer, E. (2017) · ACM Transactions on Graphics · doi:10.1145/3137609 · Cited by 150
Multi-view to Novel View: Synthesizing Novel Views With Self-learned Confidence
Sun, S.-H., Huh, M., Liao, Y.-H., Zhang, N., & Lim, J. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_10 · Cited by 149
Carved Visual Hulls for Image-Based Modeling
Furukawa, Y., & Ponce, J. (2006) · Lecture notes in computer science · doi:10.1007/11744023_44 · Cited by 127
PixelNet: Representation of the pixels, by the pixels, and for the pixelsPreprint
Bansal, A., Chen, X., Russell, B. C., Gupta, A. K., & Ramanan, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06506 · Cited by 126
Deep Volumetric Video From Very Sparse Multi-view Performance Capture
Huang, Z., Li, T., Chen, W., Zhao, Y., Xing, J., LeGendre, C., Luo, L., Ma, C., & Li, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_21 · Cited by 123
Self-citationScalable 3D video of dynamic scenes
Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., & Groß, M. H. (2005) · The Visual Computer · doi:10.1007/s00371-005-0346-7 · Cited by 93
Estimation of Human Body Shape in Motion with Wide Clothing
Yang, J., Franco, J.-S., Hétroy, F., & Wuhrer, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_27 · Cited by 85
3D Human Body Reconstruction from a Single Image via Volumetric Regression
Jackson, A. S., Manafas, C., & Tzimiropoulos, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_6 · Cited by 79
Human pose estimation from silhouettes: a consistent approach using distance level sets
Sminchisescu, C., & Telea, A. C. (2002) · TU/e Research Portal · Cited by 73
Volumetric Performance Capture from Minimal Camera Viewpoints
Gilbert, A., Volino, M., Collomosse, J. P., & Hilton, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_35 · Cited by 49
Cited by
pixelNeRF: Neural Radiance Fields from One or Few Images
Yu, A., Ye, V., Tancik, M., & Kanazawa, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00455 · Cited by 1,560
Self-citationFourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459
VIBE: Video Inference for Human Body Pose and Shape Estimation
Kocabas, M., Athanasiou, N., & Black, M. J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00530 · Cited by 1,031
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 854
Self-citationDepth-supervised NeRF: Fewer Views and Faster Training for Free
Deng, K., Liu, A., Zhu, J.-Y., & Ramanan, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01254 · Cited by 841
Metadata
- Authors
- Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2304-2314
- OpenAlex
- W2981978060
- Cited by
- 1,228
- References
- 66
- FWCI
- 165.18
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.