The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Fusion4D: real-time performance capture of challenging scenes
Open accessAbstract
We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well as parameterizing the nonrigid scene motion. Our approach is highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes. We demonstrate advantages over related real-time techniques that either deform an online generated template or continually fuse depth data nonrigidly into a single reference model. Finally, we show geometric reconstruction results on par with offline methods which require orders of magnitude more processing time and many more RGBD cameras.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 2×
“Following the nonrigid alignment we then fuse the data from the current frame into the reference volumeV as in [Newcombe et al. 2011].”
BackgroundMethodSelf-citationEfficient variants of the ICP algorithm
Rusinkiewicz, S., & Levoy, M. (2002) · Proceedings Third International Conference on 3-D Digital Imaging and Modeling · doi:10.1109/im.2001.924423 · Cited by 3,945
The Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542
Object modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
A Theory of Shape by Space Carving
Kutulakos, K. N., & Seitz, S. M. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008191222954 · Cited by 1,254
Fusing points and lines for high performance tracking
Rosten, E., & Drummond, T. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.104 · Cited by 1,119
DeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.175 · Cited by 997
DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
Newcombe, R. A., Fox, D., & Seitz, S. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298631 · Cited by 951
Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.377 · Cited by 930
Self-citationPatchMatch Stereo - Stereo Matching with Slanted Support Windows
Bleyer, M., Rhemann, C., & Rother, C. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.14 · Cited by 736
Self-citationEpicFlow: Edge-preserving interpolation of correspondences for optical flow
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298720 · Cited by 697
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360696 · Cited by 610
High-quality streamable free-viewpoint video
Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., & Sullivan, S. (2015) · ACM Transactions on Graphics · doi:10.1145/2766945 · Cited by 573
Performance capture from sparse multi-view video
de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., & Thrun, S. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360697 · Cited by 535
Surface Capture for Performance-Based Animation
Starck, J., & Hilton, A. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.68 · Cited by 452
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449
Real-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405
Self-citationEmbedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276478 · Cited by 337
Robust single-view geometry and motion reconstruction
Li, H., Adams, B., Guibas, L. J., & Pauly, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618521 · Cited by 334
Scalable real-time volumetric surface reconstruction
Chen, J., Bautembach, D., & Izadi, S. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461940 · Cited by 247
Self-citationDynamic shape capture using multi-view photometric stereo
Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618520 · Cited by 238
High-quality passive facial performance capture using anchor frames
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., & Gross, M. (2011) · ACM SIGGRAPH 2011 papers · doi:10.1145/1964921.1964970 · Cited by 238
High-quality passive facial performance capture using anchor frames
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., & Gross, M. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964970 · Cited by 223
A Survey on Human Motion Analysis from Depth Data
Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., & Gall, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-44964-2_8 · Cited by 213
Fast articulated motion tracking using a sums of Gaussians body model
Stoll, C., Hasler, N., Gall, J., Seidel, H.-P., & Theobalt, C. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126338 · Cited by 211
Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences
Bogo, F., Black, M. J., Loper, M., & Romero, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.265 · Cited by 200
Articulated mesh animation from multi-view silhouettes
Vlasic, D., Baran, I., Matusik, W., & Popović, J. (2008) · ACM SIGGRAPH 2008 papers · doi:10.1145/1399504.1360696 · Cited by 187
Dense Human Body Correspondences Using Convolutional Networks
Wei, L., Huang, Q., Ceylan, D., Vouga, E., & Li, H. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.171 · Cited by 185
Bradley, D., Popa, T., Sheffer, A., Heidrich, W., & Boubekeur, T. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360698 · Cited by 176
Real-Time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera
Ye, M., & Yang, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.301 · Cited by 164
3D scanning deformable objects with a single RGBD sensor
Dou, M., Taylor, J., Fuchs, H., Fitzgibbon, A. W., & Izadi, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298647 · Cited by 154
Self-citation3D video and free viewpoint video—From capture to display
Smolić, A. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.005 · Cited by 144
Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., & Schilling, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1516522.1516526 · Cited by 136 · cited 2×
“Other nonrigid tech-121 niques remove the need for a shape or template prior, but assume small and smooth motions [Zeng et al. 2013; Wand et al. 2009; Mitra et al. 2007]; or deal with topology changes in the input data (e.g., the fusing and then separation of hands) but suffer from drift and…”
BackgroundMonoFusion: Real-time 3D reconstruction of small scenes with a single web camera
Pradeep, V., Rhemann, C., Izadi, S., Zach, C., Bleyer, M., & Bathiche, S. (2013) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2013.6671767 · Cited by 131
Self-citationRobust Non-rigid Motion Tracking and Surface Reconstruction Using L0 Regularization
Guo, K., Xu, F., Wang, Y., Liu, Y., & Dai, Q. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.353 · Cited by 122
Robust Bundle Adjustment Revisited
Zach, C. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_50 · Cited by 109
Scalable 3D video of dynamic scenes
Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., & Groß, M. H. (2005) · The Visual Computer · doi:10.1007/s00371-005-0346-7 · Cited by 107
Motion capture using joint skeleton tracking and surface estimation
Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., & Seidel, H. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206755 · Cited by 101
Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L. J., & Pottmann, H. (2007) · doi:10.5555/1281991.1282015 · Cited by 101
Quality Dynamic Human Body Modeling Using a Single Low-Cost Depth Camera
Zhang, Q., Fu, B., Ye, M., & Yang, R. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.92 · Cited by 96
Free-form mesh tracking: A patch-based approach
Cagniart, C., Boyer, E., & Ilić, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539814 · Cited by 91
Animation cartography—intrinsic reconstruction of shape and motion
Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., & Seidel, H. (2012) · ACM Transactions on Graphics · doi:10.1145/2159516.2159517 · Cited by 90
Scanning and tracking dynamic objects with commodity depth cameras
Dou, M., Fuchs, H., & Frahm, J. (2013) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2013.6671769 · Cited by 76
Self-citationTemplateless Quasi-rigid Shape Modeling with Implicit Loop-Closure
Zeng, M., Zheng, J., Cheng, X., & Liu, X. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.26 · Cited by 72
Tracking surfaces with evolving topology
Bojsen-Hansen, M., Li, H., & Wojtan, C. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185549 · Cited by 60
Metric Regression Forests for Correspondence Estimation
Pons‐Moll, G., Taylor, J., Shotton, J., Hertzmann, A., & Fitzgibbon, A. W. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0818-9 · Cited by 59
Self-citationFilter-Based Mean-Field Inference for Random Fields with Higher-Order Terms and Product Label-Spaces
Vineet, V., Warrell, J., & Torr, P. H. S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33715-4_3 · Cited by 45
Wang, S., Fanello, S., Rhemann, C., Izadi, S., & Kohli, P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.21 · Cited by 44
Self-citationPerformance Capture from Multi-View Video
Theobalt, C., de Aguiar, E., Stoll, C., Seidel, H., & Thrun, S. (2010) · Geometry and computing · doi:10.1007/978-3-642-12392-4_6 · Cited by 41
Cited by
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
VNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Li, Z., Niklaus, S., Snavely, N., & Wang, O. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00643 · Cited by 674
Holoportation: Virtual 3D Teleportation in Real-time
Orts-Escolano, S., Rhemann, C., Fanello, S., Chang, W., Kowdle, A., Degtyarev, Y., Kim, D., Davidson, P. L., Khamis, S., Dou, M., et al. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984517 · Cited by 662
Self-citation
Metadata
- Authors
- Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., Kohli, P., Tankovich, V., & Izadi, S.
- Venue
- ACM Transactions on Graphics
- Published
- 2016-07-11
- Volume
- 35
- Issue
- 4
- Pages
- 1-13
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2461005315
- Cited by
- 500
- References
- 52
- FWCI
- 46.92
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.