The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
DoubleFusion: Real-Time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor
Abstract
We propose DoubleFusion, a new real-time system that combines volumetric dynamic reconstruction with data-driven template fitting to simultaneously reconstruct detailed geometry, non-rigid motion and the inner human body shape from a single depth camera. One of the key contributions of this method is a double layer representation consisting of a complete parametric body shape inside, and a gradually fused outer surface layer. A pre-defined node graph on the body surface parameterizes the non-rigid deformations near the body, and a free-form dynamically changing graph parameterizes the outer surface layer far from the body, which allows more general reconstruction. We further propose a joint motion tracking method based on the double layer representation to enable robust and fast motion tracking performance. Moreover, the inner body shape is optimized online and forced to fit inside the outer surface layer. Overall, our method enables increasingly denoised, detailed and complete surface reconstructions, fast motion tracking performance and plausible inner body shape reconstruction in real-time. In particular, experiments show improved fast motion tracking and loop closure performance on more challenging scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & J, B. M. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818013 · Cited by 3,734
Self-citationKeep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., & Black, M. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_34 · Cited by 1,498
SCAPE: shape completion and animation of people
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073207 · Cited by 1,489
DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
Newcombe, R. A., Fox, D., & Seitz, S. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298631 · Cited by 951
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360696 · Cited by 610
Fusion4D: real-time performance capture of challenging scenes
Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925969 · Cited by 500
ClothCap: seamless 4D clothing capture and retargeting
Pons-Moll, G., Pujades, S., Hu, S., & Black, M. J. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073711 · Cited by 495
Self-citationVideo Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
Self-citationEmbedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449
Real-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405
Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans
Li, H., Sumner, R. W., & Pauly, M. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01282.x · Cited by 399
Self-citationMotion capture using joint skeleton tracking and surface estimation
Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., & Seidel, H.-P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206755 · Cited by 344
Dyna: a model of dynamic human shape in motion
Pons-Moll, G., Romero, J., Mahmood, N., & Black, M. J. (2015) · ACM Transactions on Graphics · doi:10.1145/2766993 · Cited by 323
Self-citationVolumeDeform: Real-Time Volumetric Non-rigid Reconstruction
Innmann, M., Zollhöfer, M., Nießner, M., Theobalt, C., & Stamminger, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_22 · Cited by 308
Detailed, Accurate, Human Shape Estimation from Clothed 3D Scan Sequences
Zhang, C., Pujades, S., Black, M., & Pons-Moll, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.582 · Cited by 295
Self-citationThe Vitruvian manifold: Inferring dense correspondences for one-shot human pose estimation
Taylor, J., Shotton, J., Sharp, T., & Fitzgibbon, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247664 · Cited by 270
Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences
Bogo, F., Black, M. J., Loper, M., & Romero, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.265 · Cited by 200
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM SIGGRAPH 2008 papers · doi:10.1145/1399504.1360696 · Cited by 187
KillingFusion: Non-rigid 3D Reconstruction without Correspondences
Slavcheva, M., Baust, M., Cremers, D., & Ilic, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.581 · Cited by 177
BodyFusion: Real-Time Capture of Human Motion and Surface Geometry Using a Single Depth Camera
Yu, T., Guo, K., Xu, F., Dong, Y., Su, Z., Zhao, J., Li, J., Dai, Q., & Liu, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.104 · Cited by 177
Self-citationBradley, D., Popa, T., Sheffer, A., Heidrich, W., & Boubekeur, T. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360698 · Cited by 176
Real-Time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera
Ye, M., & Yang, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.301 · Cited by 164
Li, H., Vouga, E., Gudym, A., Luo, L., Barron, J. T., & Gusev, G. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508407 · Cited by 162
Self-citationMarkerless motion capture of interacting characters using multi-view image segmentation
Liu, Y., Stoll, C., Gall, J., et al. (2011) · doi:10.1109/cvpr.2011.5995424 · Cited by 157
Self-citationReal-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera
Guo, K., Xu, F., Yu, T., Liu, X., Dai, Q., & Liu, Y. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3083722 · Cited by 147
Self-citationCombined Region and Motion-Based 3D Tracking of Rigid and Articulated Objects
Brox, T., Rosenhahn, B., Gall, J., & Cremers, D. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.32 · Cited by 143
A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video
Liu, Y., Dai, Q., & Xu, W. (2009) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2009.88 · Cited by 137
Self-citationEfficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., & Schilling, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1516522.1516526 · Cited by 136 · cited 1×
“…many meth-ods have been proposed to address free-form capture: linear variational deformation [20], deformation graph [18], sub-space deformation [40], articulate deformation [7, 8] and [29], 4D spatio-temporal surface [26] and [36], incompress-ible flows [33], animation cartography [38],…”
BackgroundMarkerless Motion Capture of Multiple Characters Using Multiview Image Segmentation
Liu, Y., Gall, J., Stoll, C., Dai, Q., Seidel, H., & Theobalt, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.47 · Cited by 136
Self-citationArticulated Object Reconstruction and Markerless Motion Capture from Depth Video
Pekelny, Y., & Gotsman, C. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01137.x · Cited by 114
Performance Capture of Interacting Characters with Handheld Kinects
Ye, G., Liu, Y., Hasler, N., Ji, X., Dai, Q., & Theobalt, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_59 · Cited by 110
Self-citationModeling deformable objects from a single depth camera
Liao, M., Zhang, Q., Wang, H., Yang, R., & Gong, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459161 · Cited by 101
Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L. J., & Pottmann, H. (2007) · doi:10.5555/1281991.1282015 · Cited by 101
Animation cartography—intrinsic reconstruction of shape and motion
Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., & Seidel, H. (2012) · ACM Transactions on Graphics · doi:10.1145/2159516.2159517 · Cited by 88
Global registration of dynamic range scans for articulated model reconstruction
Chang, W., & Zwicker, M. (2011) · ACM Transactions on Graphics · doi:10.1145/1966394.1966405 · Cited by 87
Reconstructing Animated Meshes from Time‐Varying Point Clouds
Süßmuth, J., Winter, M., & Greiner, G. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01287.x · Cited by 81
Robust single-view geometry and motion reconstruction
Li, H., Adams, B., Guibas, L. J., & Pauly, M. V. (2009) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/1661412.1618521 · Cited by 76
Self-citationSpace-time surface reconstruction using incompressible flow
Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., & Cohen‐Or, D. (2008) · ACM Transactions on Graphics · doi:10.1145/1409060.1409063 · Cited by 73
Scanning and tracking dynamic objects with commodity depth cameras
Dou, M., Fuchs, H., & Frahm, J. (2013) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2013.6671769 · Cited by 71
Range Scan Registration Using Reduced Deformable Models
Chang, W., & Zwicker, M. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01384.x · Cited by 71
Multi-view Dynamic Shape Refinement Using Local Temporal Integration
Leroy, V., Franco, J.-S., & Boyer, E. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.336 · Cited by 65
General Dynamic Scene Reconstruction from Multiple View Video
Mustafa, A., Kim, H., Guillemaut, J., & Hilton, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.109 · Cited by 61
Bradley, D., Popa, T., Sheffer, A., Heidrich, W., & Boubekeur, T. (2008) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1399504.1360698 · Cited by 54
Metric Regression Forests for Correspondence Estimation
Pons‐Moll, G., Taylor, J., Shotton, J., Hertzmann, A., & Fitzgibbon, A. W. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0818-9 · Cited by 53
Self-citationRealtime Reconstruction of an Animating Human Body from a Single Depth Camera
Chen, Y., Cheng, Z., Lai, C., Martin, R. R., & Dang, G. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2478779 · Cited by 39
Cited by
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion
Chibane, J., Alldieck, T., & Pons-Moll, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00700 · Cited by 509
Self-citationFunction4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors
Yu, T., Zheng, Z., Guo, K., Liu, P., Dai, Q., & Liu, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00569 · Cited by 355
Self-citationARCH: Animatable Reconstruction of Clothed Humans
Huang, Z., Xu, Y., Lassner, C., Li, H., & Tung, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00316 · Cited by 341
Self-citation
Metadata
- Authors
- Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., Pons-Moll, G., & Liu, Y.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 7287-7296
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964219767
- Cited by
- 255
- References
- 46
- FWCI
- 49.53
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.