DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors. Our DynamicFusion approach reconstructs scene geometry whilst simultaneously estimating a dense volumetric 6D motion field that warps the estimated geometry into a live frame. Like KinectFusion, our system produces increasingly denoised, detailed, and complete reconstructions as more measurements are fused, and displays the updated model in real time. Because we do not require a template or other prior scene model, the approach is applicable to a wide range of moving objects and scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the SLAM community, Newcombe et al. [182] have address the non-rigid case for small-scale reconstruction.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [182]
Citations
Cites
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 14×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationA volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 3×
“The truncation distance τ > 0 and the updated TSDF value is given by the weighted averaging scheme [5], with the weight truncation introduced in [18]:”
BackgroundMethodKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 13×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationReal-time 3D reconstruction at scale using voxel hashing
Nießner, M., Zollhöfer, M., Izadi, S., & Stamminger, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508374 · Cited by 1,002 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundEfficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundKintinuous: Spatially Extended KinectFusion
Whelan, T. J., Kaess, M., Fallon, M. F., Johannsson, H., Leonard, J. J., & McDonald, J. (2012) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 624
Realtime and Robust Hand Tracking from Depth
Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.145 · Cited by 470
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449 · cited 3×
“Hierarchical Deformation Tree: In original applications of the embedded deformation graph [25] approach to non-rigid tracking, E is defined as the k−nearest neighbours of each node or all nodes within a specified radius.”
BackgroundInfluentialReal-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405 · cited 1×
“Recently, [37] demonstrated an impressive real-time version of a similar technique, using GPU accelerated optimisations.”
MethodGlobal Correspondence Optimization for Non‐Rigid Registration of Depth Scans
Li, H., Sumner, R. W., & Pauly, M. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01282.x · Cited by 399 · cited 1×
“Practical advancements to pairwise 3D shape and scan alignment over larger deformations make use of reduced deformable model parametrisations [14, 4].”
Method3D shape regression for real-time facial animation
Cao, C., Weng, Y., Lin, S., & Zhou, K. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462012 · Cited by 348 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundEmbedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276478 · Cited by 337
Robust single-view geometry and motion reconstruction
Li, H., Adams, B., Guibas, L. J., & Pauly, M. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618521 · Cited by 334
Realtime facial animation with on-the-fly correctives
Li, H., Yu, J., Ye, Y., & Bregler, C. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462019 · Cited by 282 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundSkinning with dual quaternions
Kavan, L., Collins, S., Žára, J., & O'Sullivan, C. (2007) · Proceedings of the 2007 symposium on Interactive 3D graphics and games · doi:10.1145/1230100.1230107 · Cited by 270 · cited 6×
“We therefore introduce the explicit warped model to live camera transform, T lw , and compose this onto the volumetric warp function; our complete warp-field is then given as: W t (x c ) = T lw SE3(DQB(x c )).”
MethodInfluentialThe Vitruvian manifold: Inferring dense correspondences for one-shot human pose estimation
Taylor, J., Shotton, J., Sharp, T., & Fitzgibbon, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247664 · Cited by 270
Roth, H., & Vona, M. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.112 · Cited by 172 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundGlobal non-rigid alignment of 3-D scans
Brown, B. J., & Rusinkiewicz, S. M. (2007) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1275808.1276404 · Cited by 170 · cited 1×
“Several researchers have extended ICP to enable small non-rigid deformations, e.g., [1] [2].”
BackgroundGlobal non-rigid alignment of 3-D scans
Brown, B. J., & Rusinkiewicz, S. M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276404 · Cited by 165
Real-Time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera
Ye, M., & Yang, R. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.301 · Cited by 164 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundLi, H., Vouga, E., Gudym, A., Luo, L., Barron, J. T., & Gusev, G. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508407 · Cited by 162 · cited 1×
“Quasi-rigid reconstruction has also been demonstrated [15, 35] and hybrid systems, making use of a known kinematic structure (e.”
MethodDART: Dense Articulated Real-Time Tracking
Schmidt, T., Newcombe, R. A., & Fox, D. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.030 · Cited by 144 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundSelf-citationEfficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., & Schilling, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1516522.1516526 · Cited by 136
Deformation-based loop closure for large scale dense RGB-D SLAM
Whelan, T. J., Kaess, M., Leonard, J. J., & McDonald, J. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696405 · Cited by 126 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundTemporally coherent completion of dynamic shapes
Li, H., Luo, L., Vlasic, D., Peers, P., Popović, J., Pauly, M. V., & Rusinkiewicz, S. M. (2012) · ACM Transactions on Graphics · doi:10.1145/2077341.2077343 · Cited by 123 · cited 1×
“Other work combines non-rigid mesh template tracking and temporal denoising and completion [13], but does not obtain a single consistent representation of the scene.”
Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L. J., & Pottmann, H. (2007) · doi:10.5555/1281991.1282015 · Cited by 97
Reconstruction of deforming geometry from time-varying point clouds
Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L. J., & Schilling, A. (2007) · doi:10.5555/1281991.1281998 · Cited by 96
Patch Volumes: Segmentation-Based Consistent Mapping with RGB-D Cameras
Henry, P., Fox, D., Bhowmik, A. K., & Mongia, R. K. (2013) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2013.59 · Cited by 96 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundSelf-citationQuality Dynamic Human Body Modeling Using a Single Low-Cost Depth Camera
Zhang, Q., Fu, B., Ye, M., & Yang, R. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.92 · Cited by 96 · cited 1×
“, a human body), are able to perform non-rigid shape denoising [36].”
BackgroundNew Methods for Surface Reconstruction from Range Images
Curless, B. (1997) · Cited by 94
Animation cartography—intrinsic reconstruction of shape and motion
Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., & Seidel, H. (2012) · ACM Transactions on Graphics · doi:10.1145/2159516.2159517 · Cited by 90
Space-time surface reconstruction using incompressible flow
Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., & Cohen‐Or, D. (2008) · ACM Transactions on Graphics · doi:10.1145/1409060.1409063 · Cited by 84
Reconstructing Animated Meshes from Time‐Varying Point Clouds
Süßmuth, J., Winter, M., & Greiner, G. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01287.x · Cited by 81 · cited 1×
“An intriguing approach to template-free non-rigid alignment, introduced in [17] and [26], treats each nonrigid scan as a view from a 4D geometric observation and performs 4D shape reconstruction.”
BackgroundScanning and tracking dynamic objects with commodity depth cameras
Dou, M., Fuchs, H., & Frahm, J. (2013) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2013.6671769 · Cited by 76 · cited 1×
“Recent work using multiple fixed kinect cameras [8] [7] demonstrates larger scale non-rigid reconstruction by densely tracking and fusing all depth map data into a novel directional distance function representation.”
BackgroundRange Scan Registration Using Reduced Deformable Models
Chang, W., & Zwicker, M. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01384.x · Cited by 74 · cited 1×
“Practical advancements to pairwise 3D shape and scan alignment over larger deformations make use of reduced deformable model parametrisations [14, 4].”
MethodTemplateless Quasi-rigid Shape Modeling with Implicit Loop-Closure
Zeng, M., Zheng, J., Cheng, X., & Liu, X. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.26 · Cited by 72 · cited 1×
“Quasi-rigid reconstruction has also been demonstrated [15, 35] and hybrid systems, making use of a known kinematic structure (e.”
MethodReal-Time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera
Ye, M., Shen, Y., Du, C., Pan, Z., & Yang, R. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2557783 · Cited by 59
A Memory-Efficient KinectFusion Using Octree
Zeng, M., Zhao, F., Zheng, J., & Liu, X. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34263-9_30 · Cited by 56 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundNon-rigid range-scan alignment using thin-plate splines
Brown, B. J., & Rusinkiewicz, S. M. (2004) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/3dpvt.2004.88 · Cited by 51
Non-rigid range-scan alignment using thin-plate splines
Brown, B. J., & Rusinkiewicz, S. M. (2004) · International Symposium on 3D Data Processing, Visualization and Transmission · doi:10.1109/tdpvt.2004.1335392 · Cited by 43
Temporally enhanced 3D capture of room-sized dynamic scenes with commodity depth cameras
Dou, M., & Fuchs, H. (2014) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2014.6802048 · Cited by 37 · cited 1×
“Recent work using multiple fixed kinect cameras [8] [7] demonstrates larger scale non-rigid reconstruction by densely tracking and fusing all depth map data into a novel directional distance function representation.”
BackgroundReconstruction of Deforming Geometry from Time-Varying Point Clouds
Wand, M., Jenke, P., Qi-xing Huang, A., Bokeloh, M., Guibas, L. J., & Schilling, A. (2007) · Max Planck Institute for Plasma Physics · doi:10.2312/sgp/sgp07/049-058 · Cited by 29
Research works printed but not linked: 1
- curless, 1997, New Methodsfor Surface Reconstructionfrom Range Images
Cited by
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“[182] have address the nonrigid case for small-scale reconstruction.”
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
“A major benefit of this line of work is that the reconstructed 3D content can be visualized by standard rendering techniques, and the approach directly generalizes to 4D capture as long as non-rigid surfaces can be reliably tracked [Dou et al. 2016; Innmann et al. 2016; Newcombe et al. 2015].”
BackgroundNerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Self-citationVNect: real-time 3D human pose estimation with a single RGB camera
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, M., Seidel, H.-P., Xu, W., Casas, D., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073596 · Cited by 1,113
“2014] and template-free reconstruction [Dou et al. 2016; Innmann et al. 2016; Newcombe et al. 2015; Orts-Escolano et al. 2016] has been demonstrated.”
Method
Metadata
- Authors
- Newcombe, R. A., Fox, D., & Seitz, S. M.
- Affiliations
- University of Washington (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 343-352
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1938204631
- Cited by
- 951
- References
- 42
- Citations by year
- 2026: 28 · 2025: 61 · 2024: 67 · 2023: 93 · 2022: 84 · 2021: 113 · 2020: 105 · 2019: 124 · 2018: 110 · 2017: 78 · 2016: 80 · 2015: 8
- FWCI
- 159.57
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- dense SLAM, real-time 3D reconstruction, KinectFusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.