The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors
Open accessAbstract
Human volumetric capture is a long-standing topic in computer vision and computer graphics. Although high-quality results can be achieved using sophisticated off-line systems, real-time human volumetric capture of complex scenarios, especially using light-weight setups, remains challenging. In this paper, we propose a human volumetric capture method that combines temporal volumetric fusion and deep implicit functions. To achieve high-quality and temporal-continuous reconstruction, we propose dynamic sliding fusion to fuse neighboring depth observations together with topology consistency. Moreover, for detailed and complete surface generation, we propose detailpreserving deep implicit functions for RGBD input which can not only preserve the geometric details on the depth inputs but also generate more plausible texturing results. Results and experiments show that our method outperforms existing methods in terms of view sparsity, generalization capacity, reconstruction quality, and run-time efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,647
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
Newcombe, R. A., Fox, D., & Seitz, S. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298631 · Cited by 951
Convolutional Occupancy Networks
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., & Geiger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_31 · Cited by 801
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Saito, S., Simon, T., Saragih, J., & Joo, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00016 · Cited by 793
Virtualized reality: constructing virtual worlds from real scenes
Kanade, T., Rander, P., & Narayanan, P. J. (1997) · IEEE Multimedia · doi:10.1109/93.580394 · Cited by 624
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360696 · Cited by 610
High-quality streamable free-viewpoint video
Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., & Sullivan, S. (2015) · ACM Transactions on Graphics · doi:10.1145/2766945 · Cited by 573
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion
Chibane, J., Alldieck, T., & Pons-Moll, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00700 · Cited by 509
Fusion4D: real-time performance capture of challenging scenes
Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925969 · Cited by 500
ClothCap: seamless 4D clothing capture and retargeting
Pons-Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495
Surface Capture for Performance-Based Animation
Starck, J., & Hilton, A. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.68 · Cited by 452
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449
Real-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Chabra, R., Lenssen, J. E., Ilg, E., Schmidt, T., Straub, J., Lovegrove, S., & Newcombe, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_36 · Cited by 380
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
Self-citationMotion capture using joint skeleton tracking and surface estimation
Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., & Seidel, H.-P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206755 · Cited by 344
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276478 · Cited by 337
Robust single-view geometry and motion reconstruction
Li, H., Adams, B., Guibas, L. J., & Pauly, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618521 · Cited by 334
Tex2Shape: Detailed Full Human Body Geometry From a Single Image
Alldieck, T., Pons-Moll, G., Theobalt, C., & Magnor, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00238 · Cited by 332
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction
Zheng, Z., Yu, T., Liu, Y., & Dai, Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050505 · Cited by 320
Self-citationKinect v2 for mobile robot navigation: Evaluation and modeling
Fankhauser, P., Bloesch, M., Rodriguez, D., Kaestner, R., Hutter, M., & Siegwart, R. (2015) · 2015 International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2015.7251485 · Cited by 314
Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., Pons-Moll, G., & Liu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00761 · Cited by 255
Self-citationDynamic shape capture using multi-view photometric stereo
Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618520 · Cited by 238
The relightables: volumetric performance capture of humans with realistic relighting
Guo, K., Lincoln, P., Davidson, P., Busch, J., Yu, X., Whalen, M., Harvey, G., Orts-Escolano, S., Pandey, R., Dourgarian, J., et al. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356571 · Cited by 236
Self-citationLiveCap: Real-Time Human Performance Capture From Monocular Video
Habermann, M., Xu, W., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2019) · ACM Transactions on Graphics · doi:10.1145/3311970 · Cited by 230
MonoPerfCap: Human Performance Capture From Monocular Video
Xu, W., Chatterjee, A., Zollhöfer, M., Rhodin, H., Mehta, D., Seidel, H.-P., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3181973 · Cited by 215
DeepCap: Monocular Human Performance Capture Using Weak Supervision
Habermann, M., Xu, W., Zollhofer, M., Pons-Moll, G., & Theobalt, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00510 · Cited by 214
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction
Bhatnagar, B. L., Sminchisescu, C., Theobalt, C., & Pons-Moll, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_19 · Cited by 200
SiCloPe: Silhouette-Based Clothed People
Natsume, R., Saito, S., Huang, Z., Chen, W., Ma, C., Li, H., & Morishima, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00461 · Cited by 199
KillingFusion: Non-rigid 3D Reconstruction without Correspondences
Slavcheva, M., Baust, M., Cremers, D., & Ilic, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.581 · Cited by 177
BodyFusion: Real-Time Capture of Human Motion and Surface Geometry Using a Single Depth Camera
Yu, T., Guo, K., Xu, F., Dong, Y., Su, Z., Zhao, J., Li, J., Dai, Q., & Liu, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.104 · Cited by 177
Self-citationBradley, D., Popa, T., Sheffer, A., Heidrich, W., & Boubekeur, T. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360698 · Cited by 176
Motion2fusion: real-time volumetric performance capture
Dou, M., Davidson, P., Fanello, S. R., Khamis, S., Kowdle, A., Rhemann, C., Tankovich, V., & Izadi, S. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130801 · Cited by 165
Detailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation
Zhu, H., Zuo, X.-X., Wang, S., Cao, X., & Yang, R.-G. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00462 · Cited by 165
Markerless motion capture of interacting characters using multi-view image segmentation
Liu, Y., Stoll, C., Gall, J., et al. (2011) · doi:10.1109/cvpr.2011.5995424 · Cited by 157
Self-citationMoulding Humans: Non-Parametric 3D Human Shape Estimation From Single Images
Gabeur, V., Franco, J.-S., Martin, X., Schmid, C., & Rogez, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00232 · Cited by 155
Combined Region and Motion-Based 3D Tracking of Rigid and Articulated Objects
Brox, T., Rosenhahn, B., Gall, J., & Cremers, D. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.32 · Cited by 152
Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera
Guo, K., Xu, F., Yu, T., Liu, X., Dai, Q., & Liu, Y. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3083722 · Cited by 151
Self-citationA Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video
Liu, Y., Dai, Q., & Xu, W. (2009) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2009.88 · Cited by 137
Self-citationMarkerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H., & Theobalt, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.47 · Cited by 136
Self-citationDeepDeform: Learning Non-Rigid RGB-D Reconstruction With Semi-Supervised Data
Božič, A., Zollhöfer, M., Theobalt, C., & NieBner, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00703 · Cited by 128
SobolevFusion: 3D Reconstruction of Scenes Undergoing Free Non-rigid Motion
Slavcheva, M., Baust, M., & Ilić, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00280 · Cited by 123
Robust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
Guo, K., Xu, F., Wang, Y., Liu, Y., & Dai, Q. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.353 · Cited by 122
Self-citationRobustFusion: Human Volumetric Capture with Data-Driven Visual Cues Using a RGBD Camera
Su, Z., Xu, L., Zheng, Z., Yu, T., Liu, Y., & Fang, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_15 · Cited by 118
Self-citationMonocular Real-Time Volumetric Performance Capture
Li, R., Xiu, Y., Saito, S., Huang, Z., Olszewski, K., & Li, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58592-1_4 · Cited by 118
Multi-View Neural Human Rendering
Wu, M., Wang, Y., Hu, Q., et al. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00175 · Cited by 114
HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs
Zheng, Z., Yu, T., Li, H., Guo, K., Dai, Q., Fang, L., & Liu, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_24 · Cited by 99
Self-citationScalable 3D video of dynamic scenes
Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., & Groß, M. H. (2005) · The Visual Computer · doi:10.1007/s00371-005-0346-7 · Cited by 93
UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction Using Commercial RGBD Cameras
Xu, L., Su, Z., Han, L., Yu, T., Liu, Y., & Fang, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2915229 · Cited by 88
Self-citationShading-based dynamic shape refinement from multi-view video under general illumination
Wu, C., Varanasi, K., Liu, Y., Seidel, H., & Theobalt, C. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126358 · Cited by 86
Self-citationTemporally Coherent 4D Reconstruction of Complex Dynamic Scenes
Mustafa, A., Kim, H., Guillemaut, J., & Hilton, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.504 · Cited by 74
Robust 3D Self-Portraits in Seconds
Li, Z., Yu, T., Pan, C., Zheng, Z., & Liu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00142 · Cited by 55
Self-citationBradley, D., Popa, T., Sheffer, A., Heidrich, W., & Boubekeur, T. (2008) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1399504.1360698 · Cited by 54
Li, C., Zhao, Z., & Guo, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_20 · Cited by 24
Cited by
Neural Fields in Visual Computing and Beyond
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., sitzmann, V., & Sridhar, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14505 · Cited by 484
Self-supervised 3D Representation Learning of Dressed Humans from Social Media Videos
Jafarian, Y., & Park, H. S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3231558 · Cited by 12
Metadata
- Authors
- Yu, T., Zheng, Z., Guo, K., Liu, P., Dai, Q., & Liu, Y.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 5742-5752
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3174025609
- Cited by
- 355
- References
- 62
- FWCI
- 43.92
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.