DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors. Our DynamicFusion approach reconstructs scene geometry whilst simultaneously estimating a dense volumetric 6D motion field that warps the estimated geometry into a live frame. Like KinectFusion, our system produces increasingly denoised, detailed, and complete reconstructions as more measurements are fused, and displays the updated model in real time. Because we do not require a template or other prior scene model, the approach is applicable to a wide range of moving objects and scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the SLAM community, Newcombe et al. [182] have address the non-rigid case for small-scale reconstruction.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [182]
Citations
Cites
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 14×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationA volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 3×
“The truncation distance τ > 0 and the updated TSDF value is given by the weighted averaging scheme [5], with the weight truncation introduced in [18]:”
BackgroundMethodKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 13×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationReal-time 3D reconstruction at scale using voxel hashing
Nießner, M., Zollhöfer, M., Izadi, S., & Stamminger, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508374 · Cited by 1,002 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundEfficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundKintinuous: Spatially Extended KinectFusion
Whelan, T. J., Kaess, M., Fallon, M. F., Johannsson, H., Leonard, J. J., & McDonald, J. (2012) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 624
Realtime and Robust Hand Tracking from Depth
Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.145 · Cited by 470
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449 · cited 3×
“Hierarchical Deformation Tree: In original applications of the embedded deformation graph [25] approach to non-rigid tracking, E is defined as the k−nearest neighbours of each node or all nodes within a specified radius.”
BackgroundInfluentialReal-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405 · cited 1×
“Recently, [37] demonstrated an impressive real-time version of a similar technique, using GPU accelerated optimisations.”
MethodGlobal Correspondence Optimization for Non‐Rigid Registration of Depth Scans
Li, H., Sumner, R. W., & Pauly, M. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01282.x · Cited by 399 · cited 1×
“Practical advancements to pairwise 3D shape and scan alignment over larger deformations make use of reduced deformable model parametrisations [14, 4].”
Method
Research works printed but not linked: 1
- curless, 1997, New Methodsfor Surface Reconstructionfrom Range Images
Cited by
Items 201–300 of 956
Dynamic polygon clouds: representation and compression for VR/AR
Pavéz, E., Chou, P. A., de Queiroz, R. L., & Ortega, A. (2018) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2018.15 · Cited by 56
DeepWrinkles: Accurate and Realistic Clothing ModelingPreprint
Lähner, Z., Cremers, D., & Tung, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.03417 · Cited by 51
“Naturally, depth information can lead to further improvement [36,14].”
BackgroundZhou, H., & Jayender, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87202-1_32 · Cited by 62
Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
Jiang, Z., Zheng, C., Laina, I., Larlus, D., & Vedaldi, A. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01921 · Cited by 74
“Classical techniques often rely on RGB-D sensors [24, 53], but such steps are impractical for many real-world scenes.”
Scene flow propagation for semantic mapping and object discovery in dynamic street scenes
Kochanov, D., Ošep, A., Stückler, J., & Leibe, B. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759285 · Cited by 58
“Recently, Newcombe et al. [12] propose DynamicFusion, a SLAM method that takes non-rigid motion in a small-scale scene into account to map a canonical shape model of the deforming object.”
MethodDeep Implicit Volume Compression
Tang, D., Singh, S., Chou, P. A., Häne, C., Dou, M., Fanello, S., Taylor, J., Davidson, P., Guleryuz, O. G., Zhang, Y., et al. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00137 · Cited by 54
Representing Volumetric Videos as Dynamic MLP Maps
Peng, S., Yan, Y., Shuai, Q., Bao, H., & Zhou, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00414 · Cited by 75
Deep Graph-based Spatial Consistency for Robust Non-rigid Point Cloud Registration
Qin, Z., Yu, H., Wang, C., Peng, Y., & Xu, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00522 · Cited by 63
Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clips
Ye, Y., Hebbar, P., Gupta, A., & Tulsiani, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01806 · Cited by 92
“On one hand, video-based methods takes in synchronized RGB(D) videos [18, 31, 64, 65, 85] or monocular videos [19, 29, 77] and fuse observation to a canonical 3D representation [48].”
MethodPmomo: Projection Mapping on Movable 3D Object
Zhou, Y., Xiao, S., Tang, N., Wei, Z., & Xu, C. G. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858329 · Cited by 53
Parallax360: Stereoscopic 360° Scene Representation for Head-Motion Parallax
Luo, B., Xu, F., Richardt, C., & Yong, J. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2794071 · Cited by 51
“Besides static scenes, dynamic objects are also able to be reconstructed with multiview input [8, 9] or single-view input [18, 28].”
BackgroundRobust Non-Rigid Registration with Reweighted Position and Transformation Sparsity
Li, K., Yang, J., Lai, Y., & Guo, D. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2832136 · Cited by 50
“Non-rigid registration is a hot research topic in computer graphics and computer vision [14], [19], [27], [32], and is a key technique for dynamic 3-”
On 3D Reconstruction Using RGB-D Cameras
Tychola, K. A., Tsimperidis, I., & Papakostas, G. A. (2022) · Digital · doi:10.3390/digital2030022 · Cited by 55
“Relatively recently, depth cameras have been developed that provide real-time 3D models.”
BackgroundTracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints
Wang, Y., McConachie, D., & Berenson, D. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561012 · Cited by 48
Novel View Synthesis of Human Interactions from Sparse Multi-view Videos
Shuai, Q., Chen, G., Fang, Q., Peng, S., Shen, W., Zhou, X., & Bao, H. (2022) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3528233.3530704 · Cited by 60
Visual Sensing and Depth Perception for Welding Robots and Their Industrial Applications
Wang, J., Li, L., & Xu, P. (2023) · Sensors · doi:10.3390/s23249700 · Cited by 51
“Subsequently, algorithms such as Dynamic Fusion [100], ReFusion [101], and Bundle Fusion [102] have emerged, further advancing the field [103].”
MethodFlyFusion: Realtime Dynamic Scene Reconstruction Using a Flying Depth Camera
Xu, L., Cheng, W., Guo, K., Han, L., Liu, Y., & Fang, L. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2930691 · Cited by 52
“Apart from building a ED graph in the key volume in the euclidean space as in [12], [14], [42] we warp the nodes to build a graph with the same topology in the data volume using the estimated local motions in Section 4.2.”
BackgroundMethodInfluentialModel-free vision-based shaping of deformable plastic materials
Cherubini, A., Ortenzi, V., Cosgun, A., Lee, R., & Corke, P. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364920907684 · Cited by 54
As-Rigid-as-Possible Shape Servoing
Shetab-Bushehri, M., Aranda, M., Mezouar, Y., & Özgür, E. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3145960 · Cited by 47
“ARAP has also been employed as deformation constraint in Shape-from-Template [24] and as regularization prior during tracking of nonrigid scenes [25], [26].”
MethodA review of research on slam technology based on the fusion of lidar and vision
Chen, P., Zhao, X., Zeng, L., Liu, L., Liu, S., Sun, L., Li, Z., Chen, H., Liu, G., Qiao, Z., et al. (2025) · Sensors · doi:10.3390/s25051447 · Cited by 46
“…While some studies have proposed solutions for dynamic environments, like Dy-naSLAM II and DS-SLAM, most SLAM systems still struggle to maintain high accuracy and robustness in highly dynamic environments, particularly when dealing with large-scale dynamic scenes and fast-moving objects [66].”
Structure and Motion from Casual Videos
Zhang, Z., Cole, F. H., Li, Z., Rubinstein, M., Snavely, N., & Freeman, W. T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19827-4_2 · Cited by 110
“One class of methods, exemplified by DynamicFusion, uses explicit depth sensing (e.g., with a Kinect sensor) to reduce the ill-posedness of the problem [22].”
MethodPair-Navi: Peer-to-Peer Indoor Navigation with Mobile Visual SLAM
Dong, E., Xu, J., Wu, C., Liu, Y., & Yang, Z. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737640 · Cited by 50
“Although some SLAM approaches attempt to reason about minimal non-rigidity with restrictive applicability [14], [16], it still remains challenging to employ visual SLAM for mobile indoor navigation in vibrant scenarios full of dynamics, such as busy shopping malls and large airports, etc. 2)…”
BackgroundLearning-Based Multi-View Stereo: A Survey
Wang, F., Zhu, Q., Chang, D., Gao, Q., Han, J., Zhang, T., Hartley, R., & Pollefeys, M. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3654665 · Cited by 56
“In addition, these sensors are active and usually consume a lot of power.”
Representing Long Volumetric Video with Temporal Gaussian Hierarchy
Xu, Z., Xu, Y., Yu, Z., Peng, S., Sun, J., Bao, H., & Zhou, X. (2024) · ACM Transactions on Graphics · doi:10.1145/3687919 · Cited by 92
“Earlier works leverage template-priors like shape-from-silhouettes [Ahmed et al. 2008], deformable models [Carranza et al. 2003] and template-free dynamic reconstruction systems with depth sensor integration [Newcombe et al. 2015].”
NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image
Wang, L., Zhao, X., Yu, T., Wang, S., & Liu, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_26 · Cited by 51
Multiresolution Deep Implicit Functions for 3D Shape Representation
Chen, Z., Zhang, Y., Genova, K., Fanello, S., Bouaziz, S., Häne, C., Du, R., Keskin, C., Funkhouser, T. A., & Tang, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01284 · Cited by 55
Towards efficient and photorealistic 3D human reconstruction: A brief survey
Chen, L., Peng, S., & Zhou, X. (2021) · Visual Informatics · doi:10.1016/j.visinf.2021.10.003 · Cited by 51
NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions
Jiang, Y., Jiang, S., Sun, G., Su, Z., Guo, K., Wu, M., Yu, J., & Xu, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00606 · Cited by 62
“Volumetric fusion based methods [34, 58, 61, 63, 69] allow free-form dynamic reconstruction in a template-free, single-view, real-time way, through updating depth into the canonical model and performing non-rigid deformation.”
BackgroundMethodSuperpixel Soup: Monocular Dense 3D Reconstruction of a Complex Dynamic Scene
Kumar, S., Dai, Y., & Li, H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2955131 · Cited by 54
“[7], [8], [9], [10] and a non-rigid object [11], [12], [13], [14], [15], in recent years, with the surge in computational resources, dense 3D reconstruction of the scene have been introduced and successfully demonstrated [1], [16], [17].”
BackgroundTexMesh: Reconstructing Detailed Human Texture and Geometry from RGB-D Video
Zhi, T., Lassner, C., Tung, T., Stoll, C., Narasimhan, S. G., & Vo, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_29 · Cited by 54
State of the Art in Dense Monocular Non‐Rigid 3D Reconstruction
Tretschk, E., Kairanda, N., Mallikarjun, B. R., Dabral, R., Kortylewski, A., Egger, B., Habermann, M., Fua, P., Theobalt, C., & Golyanik, V. (2023) · Computer Graphics Forum · doi:10.1111/cgf.14774 · Cited by 59
SparseFusion: Dynamic Human Avatar Modeling From Sparse RGBD Images
Zuo, X., Wang, S., Zheng, J., Yu, W., Gong, M., Yang, R., & Cheng, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3001506 · Cited by 52
“There are some methods [7]–[11] that use only a single depth sensor for the non-rigid objects reconstruction.”
MethodGeometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering
Chen, M., Zhang, J., Xu, X., Liu, L., Cai, Y., Feng, J., & Yan, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20050-2_14 · Cited by 59
Using extended measurements and scene merging for efficient and robust point cloud registration
Serafin, J., & Grisetti, G. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.03.008 · Cited by 42
“More recently other full graph-based SLAM systems have been proposed like Kintinuous [39], DynamicFusion [40], RTABSLAM [41] and ORB-SLAM [42].”
Real-time 3D reconstruction techniques applied in dynamic scenes: A systematic literature review
Ingale, A. K., & Udayan, J. D. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100338 · Cited by 49
TetraTSDF: 3D Human Reconstruction From a Single Image With a Tetrahedral Outer Shell
Onizuka, H., Hayirci, Z., Thomas, D., Sugimoto, A., Uchiyama, H., & Taniguchi, R.-I. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00605 · Cited by 47
Neural Free-Viewpoint Performance Rendering under Complex Human-object Interactions
Sun, G., Chen, X., Chen, Y., Pang, A., Lin, P., Jiang, Y., Xu, L., Yu, J., & Wang, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475442 · Cited by 48
“Volumetric fusion based methods [38, 70, 72, 79] enables free-form dynamic recon-”
MethodCorrespondence-Free Non-Rigid Point Set Registration Using Unsupervised Clustering Analysis
Zhao, M., Jiang, J., Ма, Л., Xin, S., Meng, G., & Dong-ming Yan, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02003 · Cited by 40
“Due to its fundamental importance, non-rigid registration plays a dominant role in a wide range of applications, such as scene reconstruction [35, 52, 54], pose tracking [34], animation [38], deformable shape manipulation and editing [42], and so on.”
Consistent depth of moving objects in video
Zhang, Z., Cole, F. H., Tucker, R., Freeman, W. T., & Dekel, T. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459871 · Cited by 63
Recent Trends in 3D Reconstruction of General Non‐Rigid Scenes
Yunus, R., Lenssen, J. E., Niemeyer, M., Liao, Y., Rupprecht, C., Theobalt, C., Pons‐Moll, G., Huang, J., Golyanik, V., & Ilg, E. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15062 · Cited by 54
“Explicit surfel representation also supports handling erroneous observations and topology change issues more efficiently.”
BackgroundPIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation
Solin, A., Cortés, S., Rahtu, E., & Kannala, J. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00073 · Cited by 43
A Survey of Simultaneous Localization and Mapping.
Huang, B., Zhao, J., & Liu, J. (2019) · Cited by 53
Context‐Aware Mixed Reality: A Learning‐Based Framework for Semantic‐Level Interaction
Chen, L., Tang, W., John, N., Wan, T.-R., & Zhang, J. J. (2019) · Computer graphics forum (Print) · doi:10.1111/cgf.13887 · Cited by 17
Spatio-Temporal Non-Rigid Registration of 3D Point Clouds of Plants
Chebrolu, N., Läbe, T., & Stachniss, C. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197569 · Cited by 43
“Approaches such as [13], [18], [20], [23], [24] aim at reconstructing scenes in an online fashion either in the presence of dynamic objects or deformations.”
Background3D Reconstruction of Non-Rigid Plants and Sensor Data Fusion for Agriculture Phenotyping
Sampaio, G. S., da Silva, L. A., & Marengoni, M. (2021) · Sensors · doi:10.3390/s21124115 · Cited by 41
“As for nonrigid 3D reconstruction, there is still no single strongly consolidated technique, but some works [46–52], such as DynamicFusion [46] and Fusion4D [48], have proposed techniques for model deformation that are shown to be very efficient.”
MethodDeep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture
Li, Y., Habermann, M., Thomaszewski, B., Coros, S., Beeler, T., & Theobalt, C. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00047 · Cited by 47
“Hence, research has shifted from expensive and complex multi-view capture setups [52, 6, 9, 7, 50, 13, 16, 38, 44, 73, 77, 50] to depth cameras [61, 21, 46, 27, 20, 35, 80, 15, 82, 81, 76, 83] over the past decade.”
BackgroundAugmented Reality 3D Discrepancy Check in Industrial Applications
Wasenmüller, O., Meyer, M., & Stricker, D. (2016) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2016.15 · Cited by 39
“[23] extended their algorithm to nonrigid scenes.”
Volumetric Capture of Humans With a Single RGBD Camera via Semi-Parametric Learning
Pandey, R., Tkach, A., Yang, S., Pidlypenskyi, P., Taylor, J., Martin-Brualla, R., Tagliasacchi, A., Papandreou, G., Davidson, P., Keskin, C., et al. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00994 · Cited by 40
“Other methods [51, 35, 12, 11, 36, 13], further push this technology by using highly customized, high speed RGBD sensors.”
BackgroundMethodBujanca, M., Gafton, P., Sajad Saeedi, G., Nisbet, A. P., Bodin, B., OaBoyle Michael, F. P., Davison, A. J., Kelly, P. H. J., Riley, G. D., Lennox, B., et al. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794369 · Cited by 46
“The insight behind DynamicFusion [45] is that SLAM systems operating under a static model of reality can suffer from localisation failures, and ultimately build corrupted or incomplete reconstructions.”
BackgroundMethodInfluentialOcclusionFusion: Occlusion-aware Motion Estimation for Real-time Dynamic 3D Reconstruction
Lin, W., Zheng, C., Yong, J., & Xu, F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00178 · Cited by 54
“On the online side, existing methods use iterative geometry fitting [29], sparse image feature matching [18], or photometric constrains [15] to estimate object motions.”
BackgroundMethodResultInfluentialWang, K., Guo, J. H., Chen, K., & Lǚ, J. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3545479 · Cited by 42
Capturing Dynamic Textured Surfaces of Moving Targets
Wang, R., Wei, L., Vouga, E., Huang, Q., Ceylan, D., Medioni, G., & Li, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_17 · Cited by 38
SCULPTOR: Skeleton-Consistent Face Creation Using a Learned Parametric Generator
Qiu, Z., Li, Y., He, D., Zhang, Q., Zhang, L., Zhang, Y., Wang, J., Xu, L., Wang, X., Zhang, Y., et al. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555462 · Cited by 43
Implicit Neural Representations With Structured Latent Codes for Human Body Modeling
Peng, S., Chen, G., Zhang, Y., Xu, Y., Wang, Q., Shuai, Q., Zhou, X., & Bao, H. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3245815 · Cited by 51
“Most methods [3], [4], [7], [32] adopt the traditional modeling and rendering pipeline to synthesize novel views of humans.”
MethodTowards a hybrid space combining Spatial Augmented Reality and virtual reality
Roo, J. S., & Hachet, M. (2017) · IEEE Symposium on 3D User Interfaces (3DUI) · doi:10.1109/3dui.2017.7893339 · Cited by 36
“Object tracking is not currently supported, yet it can be implemented using off the shelf solutions such as Optitrack (taking into account infrared interference from the current sensors), or markerless tracking [17].”
BackgroundHeadFusion: 360 Head Pose Tracking Combining 3D Morphable Model and 3D Reconstruction
Yu, Y., Mora, K. A. F., & Odobez, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2841403 · Cited by 40
“This approach has recently been extended via DynamicFusion [34] to also handle non-rigid objects through the estimation of a dense nonrigid warp field.”
MethodFarfetchFusion: Towards Fully Mobile Live 3D Telepresence Platform
Lee, K., Yi, J., & Lee, Y. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3592525 · Cited by 43
“Among the vast design space, we first build the base pipeline based on the volumetric fusion [24, 39, 40], a representative 3D reconstruction approach that generates realistic and smooth surfaces from multiview RGB-D inputs.”
MethodInfluentialA Continuous Optimization Approach for Efficient and Accurate Scene Flow
Lv, Z., Beall, C., Alcantarilla, P. F., Li, F., Kira, Z., & Dellaert, F. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_46 · Cited by 38
“[13,15,26], video analysis [7], dynamic reconstruction [12,14], etc.”
Scaling CNNs for High Resolution Volumetric Reconstruction from a Single Image
Johnston, A., Garg, R., Carneiro, G., Reid, I. D., & van den Hengel, A. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.114 · Cited by 43
“Systems like KinectFusion [25] and DynamicFusion [24] have opened the possibilities for various applications in the field of augmented and virtual reality by providing high quality reconstruction with the help of cheap sensors like Kinect.”
BackgroundTemporally Consistent Online Depth Estimation in Dynamic Scenes
Li, Z., Ye, W., Wang, D., Creighton, F. X., Taylor, R. H., Venkatesh, G., & Unberath, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00303 · Cited by 45
“Many approaches [8, 16, 21, 31], such as DynamicFusion, accumulate information over all past frames.”
BackgroundDELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image
Li, Y., Liu, X., Dong, W., Zhou, H., Bao, H., Zhang, G., Zhang, Y., & Cui, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19769-7_36 · Cited by 49
AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capture
Li, Z., Zheng, Z., Zhang, H., Ji, C., & Liu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19769-7_19 · Cited by 34
FlashFusion: Real-time Globally Consistent Dense 3D Reconstruction using CPU Computing
Han, L., & Fang, L. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.006 · Cited by 35
“Consider the relatively higher universality of voxel-based techniques to fit in with various applications including path planning [24], machine perception [23] and dynamic reconstruction [14], we mainly focus on voxel-based methods in this paper.”
3D Face From X: Learning Face Shape From Diverse Sources
Guo, Y., Cai, L., & Zhang, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3065798 · Cited by 39
“Reference [42] proposes a direct approach that reconstructs and tracks the 3D face without templates or prior models.”
BackgroundSelf-supervised Neural Articulated Shape and Appearance Models
Wei, F., Chabra, R., Ma, L., Lassner, C., Zollhöfer, M., Rusinkiewicz, S. M., Sweeney, C., Newcombe, R. A., & Slavcheva, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01536 · Cited by 63
“DynamicFusion [38] relied on local depth correspondences, follow-up methods used sparse SIFT features [20], dense color tracking [16] or dense SDF alignment [54,55].”
MethodSelf-citationReal-time visual tracking based on improved perceptual hashing
Fei, M., Ju, Z., Zhen, X., & Li, J. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3723-5 · Cited by 39
“Recently, researchers have proposed numerous algorithms to this fundamental subject in the literature [30, 34, 40, 45, 46].”
BackgroundA LINE-MOD-based markerless tracking approachfor AR applications
Wang, Y., Zhang, S., Yang, S., He, W., Bai, X., & Zeng, Y. (2016) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-016-9180-5 · Cited by 35
“More recently, large numbers of impressive tracking methods have been proposed, for example, Newcombe et al. [31, 32] developed systems (KinectFusion, DynamicFusion) that joint tracking of sensor pose and mapping of scene geometry using RGB-D sensor, the sensor pose was obtained by tracking the…”
BackgroundErgodic Exploration Using Binary Sensing for Nonparametric Shape Estimation
Abraham, I., Prabhakar, A., Hartmann, M. J. Z., & Murphey, T. D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2654542 · Cited by 39
“The idea of utilizing free space is often found in other related works of pose estimation and tracking [27]–[30] and is emphasized in our work.”
BackgroundEstimating Surface Normals with Depth Image Gradients for Fast and Accurate Registration
Nakagawa, Y., Uchiyama, H., Nagahara, H., & Taniguchi, R.-I. (2015) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2015.80 · Cited by 41
“Notably, reconstructing and tracking dynamic non-rigid surfaces now run in online and real time [12], which was hard to be achieved with existing methods.”
BackgroundReal-time compression and streaming of 4D performances
Tang, D., Dou, M., Lincoln, P., Davidson, P., Guo, K., Taylor, J., Fanello, S., Keskin, C., Kowdle, A., Bouaziz, S., et al. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275096 · Cited by 37
“2017] of traditional fusion paradigms [Dou et al. 2016; Innmann et al. 2016; Newcombe et al. 2015] that aggregate observations from all cameras into a single representation.”
BackgroundHabermann, M., Xu, W., Zollhöfer, M., Pons‐Moll, G., & Theobalt, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3093553 · Cited by 39
“Capturing general non-rigidly deforming scenes [66], [67], even at real-time frame rates [67], [68], [69], is feasible, but only works reliably for small, controlled, and slow motions.”
Lattice-Based Shape Tracking and Servoing of Elastic Objects
Shetab-Bushehri, M., Aranda, M., Mezouar, Y., & Özgür, E. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3331596 · Cited by 33
“(iii) Many of the presented tracking methods in the literature are specified to a particular object form including linear [44] or thin-shell [32]–[34], [42], [45], [46].”
BackgroundDing, H., Seenivasan, L., Killeen, B. D., Cho, S. M., & Unberath, M. (2024) · Artificial Intelligence Surgery · doi:10.20517/ais.2024.16 · Cited by 41
Chi, C., Wang, Q., Hao, T., Guo, P., & Yang, X. Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00249 · Cited by 36
“∗ represents the corresponding author Jointly estimating optical flow, stereo depth and camera motion can be applied in a wide range of applications, such as autonomous navigation [9, 42], 3D scene reconstruction [38] and robot control [1].”
BackgroundFast and Robust Non-Rigid Registration Using Accelerated Majorization-Minimization
Yao, Y., Deng, B., Xu, W., & Zhang, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3247603 · Cited by 40
“Theregistrationisperformedbyalternatelyupdating thecorrespondencesandsolvingthetransformationontheGPU toproducereal-timeresults[32],[33],[34],[35],[36].”
Zhang, H., Bo, Z.-H., Yong, J., & Xu, F. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322998 · Cited by 46
“Prior to solving the energy function, we first extract a surface mesh M from the current static model S , and the node graph G is obtained via graph generation or graph updating (Details can be found in [Newcombe et al. 2015]).”
MethodInfluentialVideo Based Reconstruction of 3D People ModelsPreprint
Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons‐Moll, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04758 · Cited by 47
“Despite remarkable progress in reconstructing 3D body models [6, 71, 81] or free-form surface [86, 44, 47, 21] from depth data, 3D reconstruction of humans in clothing from monocular video (without a pre-recorded scan of the person) has not been addressed before.”
BackgroundNeural Non-Rigid TrackingPreprint
Božič, A., Palafox, P. R., Zollhöfer, M., Dai, A., Thies, J., & Nießner, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13240 · Cited by 67
“Figure 3: Qualitative comparison of our method with DynamicFusion [23] and DeepDeform [4] on test sequences from [4].”
BackgroundMethodInfluentialRobust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos
Jiang, Y., Shen, Z., Hong, Y., Guo, C., Wu, Y., Zhang, Y., Jingyi Yu, A., & Xu, L. (2024) · ACM Transactions on Graphics · doi:10.1145/3687926 · Cited by 31
“Starting with the pioneering work DynamicFusion [Newcombe et al. 2015], which benefits from the GPU solver to achieve real-time capture, VolumeDeform [Innmann et al. 2016] combines depth-based correspondences with sparse SIFT features to reduce drift.”
Fast Texture Mapping Adjustment via Local/Global Optimization
Li, W., Gong, H., & Yang, R. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2831220 · Cited by 31
“to the more recent work of fusion of dynamic objects over space and time [5], [6], good quality 3D models can be obtained even with a single hand-held depth camera.”
BackgroundAutomatic Segmentation of Tree Structure From Point Cloud Data
Digumarti, S. T., Nieto, J. I., Cadena, C., Siegwart, R., & Beardsley, P. A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2849499 · Cited by 31
“While technology for the reconstruction of urban and humanmade structures is increasingly mature [6], [7], the same algorithms do not readily translate to the reconstruction of vege-”
BackgroundMulayCap: Multi-Layer Human Performance Capture Using a Monocular Video Camera
Su, Z., Wan, W., Yu, T., Liu, L., Fang, L., Wang, W., & Liu, Y. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3027763 · Cited by 35
“start from the fusion of a general dynamic scene [29], and have been improved by considering texture constraints [30],”
BackgroundSelf-Supervised Human Depth Estimation From Monocular Videos
Tan, F., Zhu, H., Cui, Z., Zhu, S., Pollefeys, M., & Tan, P. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00073 · Cited by 37
Animatable Neural Radiance Fields for Human Body Modeling.
Peng, S., Dong, J., Wang, Q., Zhang, S., Shuai, Q., Bao, H., & Zhou, X. (2021) · Cited by 43
ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References
He, Y., Pang, A., Chen, X., Liang, H., Wu, M., Ma, Y., & Xu, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01124 · Cited by 33
“The most handy monocular approaches for general non-rigid scenes [46, 26, 21, 57, 58, 59, 70] can only capture small, controlled, and slow motions.”
BackgroundHuman Performance Capture from Monocular Video in the Wild
Guo, C., Chen, X., Song, J., & Hilliges, O. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00097 · Cited by 37
“Depth-based approaches [37, 9, 10] reconstructs the human shape by fusing depth measurements across time, in order to filter sensor noise and complete occluded regions.”
BackgroundSurvey of maps of dynamics for mobile robots
Kucner, T. P., Magnusson, M., Mghames, S., Palmieri, L., Verdoja, F., Swaminathan, C. S., Krajník, T., Schaffernicht, E., Bellotto, N., Hanheide, M., et al. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231190428 · Cited by 42
“The work by Newcombe et al. (2015) provides a method to estimate a 6D motion fi eld, which is used to warp the estimated geometry into the live frame.”
BackgroundMethodRobust 3/6 DoF self-localization system with selective map update for mobile robot platforms
Costa, C. M., Sobreira, H., Sousa, A. J., & Veiga, G. M. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2015.09.030 · Cited by 33
“On a more smaller scale, the work presented in [16] showed impressive mapping results even with dynamic and deformable objects.”
Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery
Reichard, D., Häntsch, D., Bodenstedt, S., Suwelack, S., Wagner, M., Kenngott, H., Müller-Stich, B. P., Maier‐Hein, L., Dillmann, R., & Speidel, S. (2017) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-017-1613-6 · Cited by 31
Capturing detailed deformations of moving human bodies
Chen, H., Park, H., Macit, K., & Kavan, L. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459792 · Cited by 32
Robust and Accurate 3D Head Pose Estimation through 3DMM and Online Head Model Reconstruction
Yu, Y., Mora, K. A. F., & Odobez, J. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.90 · Cited by 36
“Although DynamicFusion [33] has been proposed recently for non-rigid objects, it is more time consuming and has some limitations like lacking face and head semantic information, or being more sensitive to fast motions.”
BackgroundTopology-Aware Non-Rigid Point Cloud Registration
Zampogiannis, K., Fermuller, C. M., & Aloimonos, Y. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2940655 · Cited by 31
Song, J., Xia, S., Wang, J., & Chen, D. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2995732 · Cited by 30
“According to [51], [52], [59], 2000 nodes are”
BackgroundMethodOn the Gap between Domestic Robotic Applications and Computational Intelligence
Zhong, J., Ling, C., Cangelosi, A., Lotfi, A., & Liu, X. (2021) · Electronics · doi:10.3390/electronics10070793 · Cited by 37
“Several algorithms have been developed for RGB-D camera-based SLAM as well, such as KinectFusion [57], Kintinuous [58], DynamicFusion [59] and so on.”
MethodVolumetric 3D Tracking by Detection
Huang, C.-H. P., Allain, B., Franco, J.-S., Navab, N., Ilić, S., & Boyer, E. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.419 · Cited by 33
“On one hand, topology changes or online surface reconstructions are better handled if surfaces are implicitly represented in volumes as e.g. truncated signed distance field (TSDF) [13, 19, 18], with high memory cost due to regular grids storing empty space information.”
BackgroundFlycon: real-time environment-independent multi-view human pose estimation with aerial vehicles
Nägeli, T., Oberholzer, S., Plüss, S., Alonso–Mora, J., & Hilliges, O. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275022 · Cited by 41
“Monocular depth cameras provide additional information and have been shown to aid robust skeletal tracking [Ganapathi et al. 2012; Shotton et al. 2013; Taylor et al. 2012] and enable dense surface reconstruction even under deformation [Dou et al. 2016; Newcombe et al. 2015; Zollhöfer et al. 2014].”
BackgroundPOSEFusion: Pose-guided Selective Fusion for Single-view Human Volumetric Capture
Li, Z., Yu, T., Zheng, Z., Guo, K., & Liu, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01394 · Cited by 33
“DynamicFusion [34] extended [17] and introduced a dense nonrigid warp field for real-time non-rigid reconstruction.”
BackgroundMethodInfluentialSu, Z., Xu, L., Zhong, D., Li, Z., Deng, F., Quan, S., & Fang, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3215746 · Cited by 40
“Apart from rigid objects, the reconstruction of non-rigid objects explored in work [11] mentioned above.”
BackgroundMethodInfluentialLu, L., Fasano, G., Carrio, A., Lei, M., Bavle, H., & Campoy, P. (2023) · Journal of Field Robotics · doi:10.1002/rob.22189 · Cited by 34
“A close similar work with KinectFusion is DynamicFusion (Newcombe et al., 2015).”
MethodInfluentialSpatiotemporal atlas parameterization for evolving meshes
Prada, F., Kazhdan, M., Chuang, M., Collet, A., & Hoppe, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073679 · Cited by 29
“Several techniques capture a performance for interactive viewing within a virtual environment [e.g., Collet et al. 2015; Dou et al. 2016; Newcombe et al. 2015; Starck and Hilton 2007; Zitnick et al. 2004].”
Metadata
- Authors
- Newcombe, R. A., Fox, D., & Seitz, S. M.
- Affiliations
- University of Washington (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 343-352
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1938204631
- Cited by
- 951
- References
- 42
- Citations by year
- 2026: 28 · 2025: 61 · 2024: 67 · 2023: 93 · 2022: 84 · 2021: 113 · 2020: 105 · 2019: 124 · 2018: 110 · 2017: 78 · 2016: 80 · 2015: 8
- FWCI
- 159.57
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- dense SLAM, real-time 3D reconstruction, KinectFusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.