DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors. Our DynamicFusion approach reconstructs scene geometry whilst simultaneously estimating a dense volumetric 6D motion field that warps the estimated geometry into a live frame. Like KinectFusion, our system produces increasingly denoised, detailed, and complete reconstructions as more measurements are fused, and displays the updated model in real time. Because we do not require a template or other prior scene model, the approach is applicable to a wide range of moving objects and scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the SLAM community, Newcombe et al. [182] have address the non-rigid case for small-scale reconstruction.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [182]
Citations
Cites
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 14×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationA volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 3×
“The truncation distance τ > 0 and the updated TSDF value is given by the weighted averaging scheme [5], with the weight truncation introduced in [18]:”
BackgroundMethodKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 13×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationReal-time 3D reconstruction at scale using voxel hashing
Nießner, M., Zollhöfer, M., Izadi, S., & Stamminger, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508374 · Cited by 1,002 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundEfficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundKintinuous: Spatially Extended KinectFusion
Whelan, T. J., Kaess, M., Fallon, M. F., Johannsson, H., Leonard, J. J., & McDonald, J. (2012) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 624
Realtime and Robust Hand Tracking from Depth
Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.145 · Cited by 470
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449 · cited 3×
“Hierarchical Deformation Tree: In original applications of the embedded deformation graph [25] approach to non-rigid tracking, E is defined as the k−nearest neighbours of each node or all nodes within a specified radius.”
BackgroundInfluentialReal-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405 · cited 1×
“Recently, [37] demonstrated an impressive real-time version of a similar technique, using GPU accelerated optimisations.”
MethodGlobal Correspondence Optimization for Non‐Rigid Registration of Depth Scans
Li, H., Sumner, R. W., & Pauly, M. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01282.x · Cited by 399 · cited 1×
“Practical advancements to pairwise 3D shape and scan alignment over larger deformations make use of reduced deformable model parametrisations [14, 4].”
Method
Research works printed but not linked: 1
- curless, 1997, New Methodsfor Surface Reconstructionfrom Range Images
Cited by
Items 501–600 of 956
Full 3D Reconstruction of Non-Rigidly Deforming Objects
Afzal, H., Aouada, D., Mirbach, B., & Ottersten, B. (2018) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3177756 · Cited by 10
“DynamicFusion is a similar work which targets real-time enhancement and incremental surface completion of non-rigidly deforming objects using a mono-view system, but suffers from similar limitations as the work by Dou and Fuchs [21, 45].”
MethodResultTajdari, F., Huysmans, T., Yao, X., XU, J., Zebarjadi, M., & Song, Y. (2024) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2024.3406645 · Cited by 17
“DynamicFusion [32] was among the early real-time methods that simultaneously tracked and reconstructed non-rigid surfaces.”
BackgroundReal-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras
Meerits, S., Nozick, V., & Saitô, H. (2017) · Journal of Real-Time Image Processing · doi:10.1007/s11554-017-0736-x · Cited by 10
“Newcombe et al. [40], Dou et al. [16] and Innmann et al. [27] support changes to the scene by deforming the reconstructed volume.”
BackgroundBaseball Player Behavior Classification System Using Long Short-Term Memory with Multimodal Features
Sun, S.-W., Mou, T.-C., Fang, C.-C., Chang, P., Hua, K., & Shih, H.-C. (2019) · Sensors · doi:10.3390/s19061425 · Cited by 10
“[9] proposed using dense simultaneous localization and mapping processes to reconstruct nonrigid scenes from RGBD information for scanning moving objects and scenes.”
BackgroundEMDQ: Removal of Image Feature Mismatches in Real-Time
Zhou, H., & Jayender, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3134456 · Cited by 10
BDIS-SLAM: a lightweight CPU-based dense stereo SLAM for surgery
Song, J., Zhang, R., Zhu, Q., Lin, J., & Ghaffari, M. (2024) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-023-03055-1 · Cited by 9
“Estimating an optimal warping field requires solving a non-linear objective function in large size and large state space, which is not possible with current hardware.”
ORB-YOLO: An Indoor IMU-aided Visual-Inertial SLAM System for Dynamic Environment
Wu, X., Miao, Y., & Sun, Z. (2023) · International Conference on Artificial Intelligence of Things and Systems (AIoTSys) · doi:10.1109/aiotsys58602.2023.00032 · Cited by 9
“Another example is DynamicFusion [10], which uses a recurrent neural network to fuse multiple frames into a consistent 3D model.”
MethodBLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling
Ramasinghe, S., Shevchenko, V., Avraham, G., & van den Hengel, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28264 · Cited by 9
“…either multiple cameras(Zhang, Curless, and Seitz 2003; Tung, Nobuhara, and Matsuyama 2009; Zhang et al. 2004; Dou, Fuchs, and Frahm 2013; Dou et al. 2016) or active depth sensors(Newcombe et al. 2011; Newcombe, Fox, and Seitz 2015; Slavcheva et al. 2017; Yu et al. 2017) to resolve the ambiguity.”
BackgroundDPF-SLAM: Dense semantic SLAM based on dynamic probability fusion in dynamic environments
Liu, X., Wen, S., Yuan, M., Li, P., Zhao, Y., & Manfredi, L. (2022) · IEEE International Conference on Real-time Computing and Robotics (RCAR) · doi:10.1109/rcar54675.2022.9872255 · Cited by 11
“In [12], [13], [14], [15], the accuracy of camera tracking”
BackgroundInteraction Control of a Robotic Manipulator With the Surface of Deformable Object
Dometios, A. C., & Tzafestas, C. S. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3226143 · Cited by 11
“It is important to distinguish deformation tracking from nonrigid reconstruction performed by methods such as those proposed in [11], [69], and [70], which reconstruct a single mesh at each frame provided by the vision sensor.”
MethodACEFusion - Accelerated and Energy-Efficient Semantic 3D Reconstruction of Dynamic Scenes
Bujanca, M., Lennox, B., & Luján, M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981591 · Cited by 13
“In contrast with volumetric methods [11], [41], [16] which need to convert between implicit and explicit representations to register incoming data with the 3D model, we build upon the efficient point-based representation of SurfelWarp [12], which can be directly registered with new input.”
BackgroundMethodLocal Selective Vision Transformer for Depth Estimation Using a Compound Eye Camera
Oh, W., Yoo, H., Ha, T., & Oh, S. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.02.010 · Cited by 9
“Among vision tasks, depth information is critical in tasks such as 3D reconstruction [8,9] and mobile robot navigation [10] .”
BackgroundButler, D. J., Keim, A. P., Ray, S., & Azim, E. (2023) · Nature Communications · doi:10.1038/s41467-023-41565-3 · Cited by 10
“Some notable examples include DynamicFusion 52 , in which depth data is used to track a non-rigid object in real time, and deformable neural irradiance fi elds 53 , where a non-rigid object is tracked and reconstructed from a series of 2D images.”
Patch-Based Non-rigid 3D Reconstruction from a Single Depth Stream
Kozlov, C., Slavcheva, M., & Ilić, S. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00016 · Cited by 9
“Single-stream Dynamic Reconstruction: The rise of single-stream non-rigid capture started with the seminal DynamicFusion method [23], which was able to incrementally track and reconstruct a non-rigid scene in real-time using dense depth-based correspondences.”
BackgroundMethodInfluentialParametric Model-Based 3D Human Shape and Pose Estimation from Multiple Views
Li, Z., Heyden, A., & Oskarsson, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20205-7_28 · Cited by 9
Taylor, C., Chris, M., McNicholas, R., & Cosker, D. P. (2019) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2019.00-22 · Cited by 10
“[21] - DynamicFusion - merges together RGBD scans to estimate a 6D motion field of a scene in real time.”
BackgroundProgressive Acquisition of SVBRDF and Shape in Motion
Ha, H., Baek, S., Nam, G., & Kim, M. H. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14087 · Cited by 9
Refining Single Low-Quality Facial Depth Map by Lightweight and Efficient Deep Model
Mu, G., Huang, D., Li, W., Hu, G., & Wang, Y. (2021) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb52358.2021.9484381 · Cited by 10
“Subsequently, DoubleFusion [8] and DynamicFusion [11] are presented, aiming to address the reconstruction of non-rigid targets.”
BackgroundMethodObject Learning for 6D Pose Estimation and Grasping from RGB-D Videos of In-hand Manipulation
Patten, T., Park, K., Leitner, M., Wolfram, K., & Vincze, M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9635884 · Cited by 12
“Global optimisation is applied to perform joint tracking before reconstruction with DynamicFusion [27].”
MethodCastellani, U., & Bartoli, A. (2020) · Springer eBooks · doi:10.1007/978-3-030-44070-1_8 · Cited by 17
Towards Implicit Correspondence in Signed Distance Field Evolution
Slavcheva, M., Baust, M., & Ilić, S. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.103 · Cited by 10
“More-over, this work has focused on implicitly tracking complete 3D models, while in future we would like to tackle dense reconstruction from a single depth stream [18], which will entail further challenges in handling partial overlap.”
BackgroundDressing 3D Humans using a Conditional Mesh-VAE-GAN.Preprint
Ma, Q., Tang, S., Pujades, S., Pons‐Moll, G., Ranjan, A., & Black, M. J. (2019) · arXiv (Cornell University) · Cited by 9
Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow
Qian, S., Zhang, G., Wu, S., & Cremers, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37517-9_4 · Cited by 12
World Reconstruction From Inconsistent Views
Höllein, L., & Nießner, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37258-1_18 · Cited by 8
iTACO: Interactable Digital Twins of Articulated Objects from Casually Captured RGBD Videos
Peng, W., Lv, J., Lu, C., & Savva, M. (2026) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv69130.2026.00056 · Cited by 22
“Signed distance functions, and later neural radiance fields and 3D gaussian splatting are commonly used [2, 6, 28, 35, 38, 43, 56, 59, 66, 69, 80].”
U4D: Unsupervised 4D Dynamic Scene Understanding
Mustafa, A., Russell, C., & Hilton, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01052 · Cited by 9
“Key-frame detection: Previous work [39, 38] showed that sparse key-frames allow robust long-term correspondence for 4D reconstruction.”
BackgroundCASIE – Computing affect and social intelligence for healthcare in an ethical and trustworthy manner
Vasiliu, L., Cortis, K., McDermott, R., Kerr, A., Peters, A., Hesse, M., Hagemeyer, J., Belpaeme, T., McDonald, J., Villing, R. C., et al. (2021) · Paladyn Journal of Behavioral Robotics · doi:10.1515/pjbr-2021-0026 · Cited by 8
“The last decade has seen research in SLAMmove towards handling dynamic environments [47], numerous different approaches have been demonstrated, such as deforming the scene in a rigid as possible method [48], estimation of joints [49], or warp fields [50].”
BackgroundFace Reconstruction Based on Multiscale Feature Fusion and 3D Animation Design
Xu, L. (2021) · Mobile Information Systems · doi:10.1155/2021/3737189 · Cited by 8
“-at is, the gradient is changed by interpolation between two 3D objects, so as to obtain the expression of the 3D model [16].”
BackgroundHolland, L. V., Stotko, P., Krumpen, S., Klein, R., & Weinmann, M. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00460 · Cited by 12
“…[67, 134, 35, 5, 183, 186, 176, 175, 24, 77, 171, 36] or additionally reconstructing the dynamics based on rigid object tracking and reconstruction [144, 172, 80, 132, 131, 50] and non-rigid object tracking and reconstruction [78, 65, 180, 107, 48, 167, 53, 167, 27, 184, 135, 56, 136, 143, 159, 8].”
BackgroundIBD-SLAM: Learning Image-Based Depth Fusion for Generalizable SLAM
Yin, M., Wu, S., & Han, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01005 · Cited by 8
Online 3D Reconstruction and Dense Tracking in Endoscopic Videos
Hayoz, M., Hahne, C., Kurmann, T., Allan, M., Beldi, G., Candinas, D., Márquez-Neila, P., & Sznitman, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72089-5_42 · Cited by 8
Inverse Rendering and Relighting From Multiple Color Plus Depth Images
Liu, S., & Do, M. N. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2728184 · Cited by 8
“Seminal works on shape refinement by fusing multiple depth maps include [4] for rigid and [5] for dynamic scenes.”
BackgroundNon-rigid Reconstruction with a Single Moving RGB-D Camera
Elanattil, S., Moghadam, P., Sridharan, S., Fookes, C., & Cox, M. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8546201 · Cited by 8
Markerless Structure-based Multi-sensor Calibration for Free Viewpoint Video Capture
Papachristou, A., Zioulis, N., Zarpalas, D., & Daras, P. (2018) · Computer Science Research Notes · doi:10.24132/csrn.2018.2801.10 · Cited by 9
“While a variety of single sensor meth-ods exist, some focusing only on geometry information [New15, Inn16, Zol14], and others also producing fully textured outputs [Guo17, Cao17], truly immersive experiences can only be facilitated by complete 360 o captures via multi-sensor systems.”
BackgroundFlexible body scanning without template models
Muñoz‐Salinas, R., Sarmadi, H., Cazzato, D., & Medina-Carnicer, R. (2018) · Signal Processing · doi:10.1016/j.sigpro.2018.09.022 · Cited by 8
3D Reconstruction System for Foot Arch Detecting Based on OpenMVG and OpenMVS
Li, Z., Luo, S., Zeng, W., Guo, S., Zhuo, J., Zhou, L., Ma, Z., & Zhang, Z. (2022) · International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · doi:10.1109/prai55851.2022.9904285 · Cited by 8
“At present, there are so many popular algorithms about 3D reconstruction, such as SFM [3], SLAM [6], KinectFusion [7], ElasticFusion [8], DynamicFusion [9], etc.”
MethodHEADSET: Human Emotion Awareness under Partial Occlusions Multimodal DataSET
Ghorbani, F., Freitas, D. R., Guillemot, C. M., Egiazarian, K., & Knorr, S. B. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3320236 · Cited by 8
“In contrast to the high number of research works in designing volumetric data generation algorithms [1,2,12,19,20,22,33,35,51] and three-dimensional (3D) reconstruction [37,38,43], there have not been enough studies in which we can discover relatively diverse and high-quality volumetric datasets…”
BackgroundCHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Images
Han, S., & Joo, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01451 · Cited by 20
“This process is inspired by the deformable object reconstruction [42] or the animatable human reconstruction pipelines [9, 10, 27, 55], where SMPL body posture provides the guidance to convert between two spaces.”
MethodLeveraging Intrinsic Properties for Non-Rigid Garment Alignment
Lin, S., Zhou, B., Zheng, Z., Zhang, H., & Liu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01332 · Cited by 8
“There has been extensive research into human performance capture that reconstructs and tracks non-rigid clothed human surface geometries [16, 36, 46, 47, 49, 59].”
BackgroundAvatarOne: Monocular 3D Human Animation
Karthikeyan, A., Ren, R., Kant, Y., & Gilitschenski, I. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00361 · Cited by 11
“Traditional human performance capture and rendering techniques often rely on multi-view videos [53, 66] or depth cameras [36, 52, 69, 74] for human body geometry reconstruction and albedo map generation.”
BackgroundSkeletonFusion: Reconstruction and tracking of human body in real-time
Xie, L., Zhang, X., Xu, Y., Shang, Y., & Yu, Q. (2018) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2018.05.011 · Cited by 8
Turning an Urban Scene Video into a Cinemagraph
Yan, H. R., Liu, Y., & Furukawa, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.177 · Cited by 8
“Dynamic Fusion reconstructs non-rigidly deforming objects from an RGBD stream, but does not focus on dynamic appearances [21].”
Background[POSTER] Semantic Augmented Reality Environment with Material-Aware Physical Interactions
Chen, L. F., Francis, K., & Tang, W. J. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar-adjunct.2017.49 · Cited by 8
Min, Z., Pan, J., Zhang, A., & Meng, M. Q. ‐. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967585 · Cited by 8
“In robotics, the aim of the dynamic fusion task is to reconstruct and track the dynamic, non-rigid scenes in real-time [5].”
BackgroundMethodSplitFusion: Simultaneous Tracking and Mapping for Non-Rigid Scenes
Li, Y., Zhang, T., Nakamura, Y., & Harada, T. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341082 · Cited by 12
“Similar to [4] [20], we represent the per-surface warping”
BackgroundMethodResultInfluentialLearning Spectral Unions of Partial Deformable 3D Shapes
Moschella, L., Melzi, S., Cosmo, L., Maggioli, F., Litany, O., Ovsjanikov, M., Guibas, L. J., & Rodolà, E. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14483 · Cited by 11
Physically Based Video Editing
Bazin, J., Kuster, C., Yu, G., Martin, T., Jacobson, A., & Groß, M. H. (2016) · Computer Graphics Forum · doi:10.1111/cgf.13039 · Cited by 8
Monocular semantic SLAM in dynamic street scene based on multiple object tracking
Chen, W., Fang, M., Liu, Y., & Li, L. (2017) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/iccis.2017.8274845 · Cited by 12
“DynamicFusion [23] uses non-grid motion to map a canonical shape model of the deforming object.”
MethodDeep Learning Based Semantic Labelling of 3D Point Cloud in Visual SLAM
Qi, X., Yang, S., & Yan, Y. (2018) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/428/1/012023 · Cited by 8
FullFusion: A Framework for Semantic Reconstruction of Dynamic Scenes
Bujanca, M., Luján, M., & Lennox, B. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00272 · Cited by 8
“Building on KinectFusion, DynamicFusion [28] introduced the first real-time non-rigid 3D reconstruction system.”
MethodInfluentialRGB-D Tracking and Optimal Perception of Deformable Objects
Cuiral-Zueco, I., & Nicolás, G. L. (2020) · IEEE Access · doi:10.1109/access.2020.3012067 · Cited by 8
“Non-rigid SLAM systems like [15] (RGB-D) and [16] (monocular) manage to solve camera location and nonrigid environment characterisation, nonetheless, they do not tackle the segmentation problem.”
BackgroundSpatiotemporal Articulated Models for Dynamic SLAMPreprint
Kumar, S., Dhiman, V., Ganesh, M. R., & Corso, J. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.03526 · Cited by 8
“A recent work has extended dense tracking and mapping to dynamic scenes by estimating a dense warp field [17].”
BackgroundMeshODE: A Robust and Scalable Framework for Mesh DeformationPreprint
Huang, J., Jiang, C. M., Leng, B., Wang, B., & Guibas, L. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11617 · Cited by 8
“Our problem is more like a non-rigid ICP problem [Li et al. 2008; Newcombe et al. 2015; Sumner et al. 2007].”
Recovering and Simulating Pedestrians in the WildPreprint
Yang, Z., Manivasagam, S., Liang, M., Yang, B., Ma, W.-C., & Urtasun, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08106 · Cited by 16
“While real-time mesh reconstruction of non-rigid objects from depth camera [29] or RGB camera [30, 31] exist, to re-articulate the humans for downstream tasks we also require human pose.”
A 3D Reconstruction and Relocalization Method for Humanoid Welding Robots
Chi, P., Wang, Z., Liao, H., Li, T., & Zhang, Q. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3544967 · Cited by 7
“Sparse mapsusefeaturepointsfor3Dcoordinates,whereasdensemaps createcomprehensivemodelsofallvisiblesurfacesandobjects, usefulforcapturingdetailedenvironmentalgeometry[24],[25], [26],[27].”
BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Video
Hong, Y., Wu, Y., Shen, Z., Guo, C., Jiang, Y., Zhang, Y., Hu, Q., Jingyi Yu, A., & Xu, L. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755233 · Cited by 7
“Building on DynamicFusion [53], subsequent methods have integrated techniques like SIFT features [29], motion field constraints [59, 60], neural deformation graphs [2], and parametric human models like SMPL [47] to address challenges in human reconstruction, including drift and human-object…”
MorpheuS: Neural Dynamic $360^{\circ}$ Surface Reconstruction from Monocular RGB-D Video
Wang, H., Wang, J., & Agapito, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01981 · Cited by 11
“This task is often approached by jointly fusing a canonical shape and a per-frame deformation field from an RGB-D video sequence [7, 8, 24, 41, 59, 60].”
MethodSPARE: Symmetrized Point-to-Plane Distance for Robust Non-Rigid 3D Registration
Yao, Y., Deng, B., Hou, J., & Zhang, J. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3598630 · Cited by 10
“Recently, DDM [42] proposes a distance metric based on directional distance fields (DDF) featuring a confidence score for enhanced alignment of 3D geometric surfaces.”
Combining Human Body Shape and Pose Estimation for Robust Upper Body Tracking Using a Depth Sensor
Probst, T., Fossati, A., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48881-3_20 · Cited by 7
“Among the model-free approaches, methods similar to KinectFusion [16, 17] perform 3D body reconstructions from RGB-D sequences or multiple views [18, 19, 11, 20, 21].”
MethodKim, Y., Lim, H., Ahn, S. C., & Kim, A. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759184 · Cited by 8
“Dense or sparse point trajectories have been illustrated by many motion estimation algorithms [10], [11], [12], [13], [14], [15], [16], and have been applied to both static and non-rigid objects.”
Background4D Human Body Correspondences from Panoramic Depth Maps
Li, Z., Wu, M., Zhou, W., & Jingyi Yu, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00304 · Cited by 10
Song, H., Huang, J., Cao, Y., & Mu, T. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0230-z · Cited by 8
“The seminal work of DynamicFusion [14] described a general pipeline adopted by many other algorithms: by parameterizing the per-frame deformation as a warp field defined on a sparse set of transformation nodes skinned from the full geometry, the underlying shape can be registered to the depth…”
MethodEstimating Muscle Activity from the Deformation of a Sequential 3D Point Cloud
Niu, H., Ito, T., Desclaux, D., Ayusawa, K., Yoshiyasu, Y., Sagawa, R., & Yoshida, E. (2022) · Journal of Imaging · doi:10.3390/jimaging8060168 · Cited by 8
“Another approach is focused on the deformation of a whole body by extending [26].”
BackgroundLearning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes
Xu, B., Davison, A. J., & Leutenegger, S. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981545 · Cited by 7
“One maps the whole changing world in a non-rigid deformable representation to deal with the changing topology of deformable/moving objects [7].”
BackgroundRecent Advances in Robot Visual SLAM
Zhang, H., Jin, H., & Ma, S. (2023) · Recent Advances in Computer Science and Communications · doi:10.2174/2666255816666230509153317 · Cited by 7
High-Resolution Volumetric Reconstruction for Clothed Humans
Tang, S., Wang, G., Ran, Q., Li, L., Shen, L., & Tan, P. (2023) · ACM Transactions on Graphics · doi:10.1145/3606032 · Cited by 9
Reconstructing 3D Face Models by Incremental Aggregation and Refinement of Depth Frames
Pala, P., & Berretti, S. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3287309 · Cited by 7
“Newcombe et al. [32] further extended the approach to cope with non-rigid objects, such as faces, but results of non-rigid object denoising were demonstrated only in cases in which the distance between the object and the camera was almost constant.”
BackgroundMethodTowards An Egocentric Framework for Rigid and Articulated Object Tracking in Virtual Reality
Taylor, C., McNicholas, R., & Cosker, D. P. (2020) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw50115.2020.00077 · Cited by 10
“On the other hand DynamicFusion [19] and VolumeDeform [6] are able to capture non-rigid behaviour in real-time from RGBD data.”
BackgroundMonocular 3D reconstruction of sail flying shape using passive markers
Maciel, L. M. D. S., Marroquim, R., Vieira, M. B., Ribeiro, K. S. D. S., & Alho, A. T. P. (2021) · Machine Vision and Applications · doi:10.1007/s00138-020-01149-3 · Cited by 7
“This constraint assumes the object deforms as minimum as possible over time and is often accompanied by an as rigid as possible constraint [5,22,23,26,27,29,38,39], which penalizes non-rigid transformations.”
BackgroundMethodDTexFusion: Dynamic Texture Fusion Using a Consumer RGBD Sensor
Zheng, C., & Xu, F. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3064846 · Cited by 7
“The acquisition of 3D geometry is critical for this purpose, and many techniques have achieved high performance in the reconstruction of either static objects[1], dynamic objects [2], or both [3].”
BackgroundMethodInfluentialVisual Tracking of Deforming Objects Using Physics-based Models
Sengupta, A., Krupa, A., & Marchand, É. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561401 · Cited by 7
“In this context, it must be noted that physical model free, RGB-D based deformation tracking approaches such as [4] depend heavily on the assumption that the deformation is isometric.”
BackgroundNormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning
Ha, H., Lee, J. H., Meuleman, A., & Kim, M. H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01571 · Cited by 7
“Real-time 3D scanning methods [15, 22, 14, 11, 8, 23] accumulate TSDFs of depth maps in the canonical space.”
BackgroundMethodRobust and Accurate 3D Self-Portraits in Seconds
Li, Z., Yu, T., Zheng, Z., & Liu, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3113164 · Cited by 8
“However, non-rigid fusion methods (e.g., [13] etc.) usually suffer from heavy drifts and error accumulation during tracking, which limit their ability to generate accurate large partial scans.”
BackgroundMethodInfluentialSelf-Supervised Depth Completion for Active Stereo
Warburg, F., Hernandez-Juarez, D., Tarrio, J., Vakhitov, A., Bonde, U., & Alcantarilla, P. F. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3145512 · Cited by 8
Advanced Visual SLAM and Image Segmentation Techniques for Augmented Reality
Jiang, Y., Tran, T. H., & Williams, L. (2022) · International Journal of Virtual and Augmented Reality · doi:10.4018/ijvar.307063 · Cited by 7
DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction With an Optimizable Feature Grid
Mohamed, M., & Agapito, L. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00046 · Cited by 7
“[7], DeepDeform [8] and DynamicFusion [46] on the shorts sequence from DeepDeform [8] dataset.”
MethodInfluentialImmersive Learning Experiences for Surgical Procedures.
Cha, Y.-W., Dou, M., Chabra, R., Menozzi, F., State, A., Wallen, E., & Fuchs, H. (2016) · PubMed · Cited by 7
Online Global Non‐rigid Registration for 3D Object Reconstruction Using Consumer‐level Depth Cameras
Xu, J., Xu, W., Yang, Y., Deng, Z., & Bao, H. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13542 · Cited by 7
Higher Order Function Networks for View Planning and Multi-View Reconstruction
Engin, S., Mitchell, E., Lee, D., Isler, V., & Lee, D. D. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197435 · Cited by 7
“With the availability of commercial depth cameras, state of the art methods such as KinectFusion [17] for rigid, and DynamicFusion [18] for nonrigid scenes have achieved great success in generating dense object reconstructions.”
BackgroundWarpingFusion: Accurate Multi-View TSDF Fusion with Local Perspective Warp
Kang, J., Lee, S., Jang, M., Yoon, H., & Lee, S. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506166 · Cited by 7
Image Based Reconstruction of Liquids from 2D Surface Detections
Richter, F., Orosco, R. K., & Yip, M. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01344 · Cited by 7
“The rigidness assumption has also been lifted to handle deformable surfaces [17, 30].”
BackgroundLearning-based Feedback Controller for Deformable Object ManipulationPreprint
Jia, B., Hu, Z., Pan, Z., Manocha, D., & Pan, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09618 · Cited by 7
“Recently, some template-less approaches are also proposed, which tackle the tracking and reconstruction problems jointly in real-time, including [43]–[45].”
BackgroundVolumeDeform: Real-time Volumetric Non-rigid ReconstructionPreprint
Innmann, M., Zollhöfer, M., Nießner, M., Theobalt, C., & Stamminger, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08161 · Cited by 7
“The depth data D t of each recorded RGB-D frame is incrementally fused into the canonical TSDF following the non-rigid fusion technique introduced in DynamicFusion [17].”
BackgroundMethodInfluentialNeural Eulerian Scene Flow FieldsPreprint
Vedder, K., Peri, N., Khatri, I., Li, S., Eaton, E., Kocamaz, M., Wang, Y., Yu, Z., Ramanan, D., & Pehserl, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.02031 · Cited by 7
“Nerfies (Park et al., 2021) and DynamicFusion (Newcombe et al., 2015) estimate a deformation field to warp a canonical frame to explain the observed frame.”
Monocular Dynamic Gaussian Splatting: Fast, Brittle, and Scene Complexity RulesPreprint
Liang, Y., Okunev, M., Uy, M. A., Li, R., Guibas, L. J., Tompkin, J., & Harley, A. W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.04457 · Cited by 7
“Earlier works utilize accurate depth cameras (Newcombe et al., 2015; Slavcheva et al., 2017) and reconstruct a dynamic scene using a monocular RGBD video.”
Global Texture Mapping for Dynamic Objects
Kim, J., Kim, H., Park, J., & Lee, S. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13872 · Cited by 6
Object-Level Semantic Map Construction for Dynamic Scenes
Kang, X., Li, J., Fan, X., Jian, H., & Xu, C. (2021) · Applied Sciences · doi:10.3390/app11020645 · Cited by 6
“One deforms the whole world in a non-rigid manner in order to include a deformable/moving object [4].”
BackgroundTowards Real-time 3D Visualization with Multiview RGB Camera Array
Ke, J., Watras, A. J., Kim, J.-J., Liu, H., Jiang, H., & Hu, Y. H. (2022) · Journal of Signal Processing Systems · doi:10.1007/s11265-021-01729-0 · Cited by 6
“Fusion4D [29] extend [27, 28] to a multiview scenario in which 8 RGB-D cameras capture depth data simultaneously, and multiple GPUs are used to compute the deformation field and the fusion of all data frame.”
BackgroundMethodA Self-Occlusion Aware Lighting Model for Real-Time Dynamic Reconstruction
Zheng, C., Lin, W., & Xu, F. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3178237 · Cited by 6
“Recently, with the development of depth sensing and parallel computing techniques, this task can be achieved by a single RGB-D sensor in real time [1], [2], [3], which may enable applications like telecommunications, VR/AR content generation, relighting, appearance editing, and so on.”
BackgroundMethodHuman Body Shape Completion with Implicit Shape and Flow Learning
Zhou, B., Meng, D., Franco, J.-S., & Boyer, E. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01240 · Cited by 7
Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical Teleconsultation
Eck, U., Wechner, M., Pankratz, F., Yu, K. K., Lazarovici, M., & Navab, N. (2023) · Applied Sciences · doi:10.3390/app131810199 · Cited by 6
iVS-Net: Learning Human View Synthesis from Internet Videos
Dong, J., Fang, Q., Yang, T., Shuai, Q., Qiao, C., & Peng, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02097 · Cited by 8
“Traditional methods rely on a dense set of cameras [6, 12, 10, 55] or depth sensors placed in the studio [33, 8, 51, 52].”
MethodA Survey of Indoor 3D Reconstruction Based on RGB-D Cameras
Zhu, J., Gao, C., Sun, Q., Wang, M., & Deng, Z. (2024) · IEEE Access · doi:10.1109/access.2024.3443065 · Cited by 8
Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos
Sun, K., Litvak, D., Zhang, Y., Li, H., Wu, J., & Wu, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_6 · Cited by 19
Sun, G., Dabral, R., Fua, P., Theobalt, C., & Habermann, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_20 · Cited by 14
Interactive Videos: Plausible Video Editing using Sparse Structure Points
Chang, C., Chu, H., & Mitra, N. J. (2016) · Computer Graphics Forum · doi:10.1111/cgf.12849 · Cited by 6
“Newcombe et al. [NFS15] propose a fully automatic system based on dense SLAM to reconstruct non-rigidly deforming scenes in real-time.”
BackgroundSHREC’19: Shape Correspondence with Isometric and Non-Isometric Deformations
Dyke, R. M., Stride, C., Lai, Y., & Rosin, P. L. (2019) · ORCA Online Research @Cardiff (Cardiff University) · doi:10.17035/d.2019.0072003316 · Cited by 9
Rasmuson, S., Sintorn, E., & Assarsson, U. (2020) · The Visual Computer · doi:10.1007/s00371-020-01823-7 · Cited by 6
“This method handles dynamic scenes by using nonrigid volumetric fusion, where new depth frames are warped to a reference model in a nonrigid manner [18].”
MethodTowards Real-Time, Multi-View Video Stereopsis
Ke, J., Watras, A. J., Kim, J.-J., Liu, H., Jiang, H., & Hu, Y. H. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054391 · Cited by 6
“RTMVS differs from existing point-cloud or 3D volumetric models [4,5,6,7,8,9,10,11,12,13] in that it uses a photo-consistent polygon surface model for both the background and moving objects.”
MethodHigh-quality First-person Rendering Mixed Reality Gaming System for In Home Setting
Chung, Y.-Y., Guo, H.-J., Kumar, H. G., & Prabhakaran, B. (2020) · IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) · doi:10.1109/aivr50618.2020.00070 · Cited by 6
“DynamicFusion [23] was simultaneously reconstructing a canonical scene and tracking a non-rigid deforming scene using TSDF in real-time without a prior model.”
Method
Metadata
- Authors
- Newcombe, R. A., Fox, D., & Seitz, S. M.
- Affiliations
- University of Washington (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 343-352
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1938204631
- Cited by
- 951
- References
- 42
- Citations by year
- 2026: 28 · 2025: 61 · 2024: 67 · 2023: 93 · 2022: 84 · 2021: 113 · 2020: 105 · 2019: 124 · 2018: 110 · 2017: 78 · 2016: 80 · 2015: 8
- FWCI
- 159.57
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- dense SLAM, real-time 3D reconstruction, KinectFusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.