DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
We present the first dense SLAM system capable of reconstructing non-rigidly deforming scenes in real-time, by fusing together RGBD scans captured from commodity sensors. Our DynamicFusion approach reconstructs scene geometry whilst simultaneously estimating a dense volumetric 6D motion field that warps the estimated geometry into a live frame. Like KinectFusion, our system produces increasingly denoised, detailed, and complete reconstructions as more measurements are fused, and displays the updated model in real time. Because we do not require a template or other prior scene model, the approach is applicable to a wide range of moving objects and scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In the SLAM community, Newcombe et al. [182] have address the non-rigid case for small-scale reconstruction.”
From Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age (2016) · cites this paper as [182]
Citations
Cites
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · cited 14×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationA volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 3×
“The truncation distance τ > 0 and the updated TSDF value is given by the weighted averaging scheme [5], with the weight truncation introduced in [18]:”
BackgroundMethodKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 13×
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationReal-time 3D reconstruction at scale using voxel hashing
Nießner, M., Zollhöfer, M., Izadi, S., & Stamminger, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508374 · Cited by 1,002 · cited 1×
“KinectFusion spurred a flurry of follow up research aimed at robustifying the tracking [9, 32] and expanding its spatial mapping capabilities to larger environments [22, 19, 34, 31, 9].”
BackgroundEfficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874 · cited 1×
“The best of these demonstrate high accuracy, real-time performance capture for tracking faces [16, 3], hands [21, 20], complete bodies [27], or general articulated objects [23, 33].”
BackgroundKintinuous: Spatially Extended KinectFusion
Whelan, T. J., Kaess, M., Fallon, M. F., Johannsson, H., Leonard, J. J., & McDonald, J. (2012) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 624
Realtime and Robust Hand Tracking from Depth
Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.145 · Cited by 470
Embedded deformation for shape manipulation
Sumner, R. W., Schmid, J., & Pauly, M. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276478 · Cited by 449 · cited 3×
“Hierarchical Deformation Tree: In original applications of the embedded deformation graph [25] approach to non-rigid tracking, E is defined as the k−nearest neighbours of each node or all nodes within a specified radius.”
BackgroundInfluentialReal-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405 · cited 1×
“Recently, [37] demonstrated an impressive real-time version of a similar technique, using GPU accelerated optimisations.”
MethodGlobal Correspondence Optimization for Non‐Rigid Registration of Depth Scans
Li, H., Sumner, R. W., & Pauly, M. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01282.x · Cited by 399 · cited 1×
“Practical advancements to pairwise 3D shape and scan alignment over larger deformations make use of reduced deformable model parametrisations [14, 4].”
Method
Research works printed but not linked: 1
- curless, 1997, New Methodsfor Surface Reconstructionfrom Range Images
Cited by
Items 301–400 of 956
MixedFusion: Real-Time Reconstruction of an Indoor Scene with Dynamic Objects
Zhang, H., & Xu, F. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2786233 · Cited by 31
“10 shows the comparison between our result and that of DynamicFusion [4], which is the state-of-the-art Fig.”
BackgroundMethodResultInfluentialTSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction
Grinvald, M., Tombari, F., Siegwart, R., & Nieto, J. I. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560923 · Cited by 35
“The pioneering work in [16] represented an important step towards dense incremental reconstruction of scenes in”
Background3D human body reconstruction based on SMPL model
Chen, D., Song, Y., Liang, F., Ma, T., Zhu, X., & Jia, T. (2022) · The Visual Computer · doi:10.1007/s00371-022-02453-x · Cited by 32
“In order to solve this problem, DynamicFusion [2] employs the SFM (Structure From Motion) strategy to reconstruct moving human bodies.”
MethodJiang, Y., Yao, K., Su, Z., Shen, Z., Luo, H., & Xu, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00065 · Cited by 41
“Besides, the monocular volumetric methods [32,41,43,58,64] obtain detailed geometry through depth fusion, while the recent advance [44] further extends it into the human-object setting.”
InfluentialHOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video
Liu, J., Cao, Y., Yang, T., Xu, Z., Keppo, J., Shan, Y., Qie, X., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01694 · Cited by 41
“Most prior approaches on dynamic scene modeling require synchronized multi-view videos [63, 21, 45, 5, 60, 53] or depth [64, 13, 31] as input.”
BackgroundAugmented Blendshapes for Real-Time Simultaneous 3D Head Modeling and Facial Motion Capture
Thomas, D., & Taniguchi, R.-I. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.359 · Cited by 28
“Recently, an extension of the popular KinectFusion algorithm [17] called DynamicFusion [18] was proposed that can handle even dynamic scenes.”
BackgroundMethodResultInfluentialSurface Reconstruction via Fusing Sparse-Sequence of Depth Images
Yang, L., Yan, Q., Fu, Y., & Xiao, C. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2657766 · Cited by 30
“Unlike the conventional optical scanner with special scanning setup [2], [3], commodity depth cameras make 3D scanning flexible and accessible to general users with low-cost [4], [5].”
BackgroundUnsupervised Learning of Consensus Maximization for 3D Vision Problems
Probst, T., Paudel, D. P., Chhatkuli, A., & Gool, L. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00102 · Cited by 28
“We assume that the transformation can be approximated by piece-wise rigidity, a well studied approximation for non-rigid surfaces [43, 28, 38, 21].”
MethodMannequinChallenge: Learning the Depths of Moving People by Watching Frozen People
Li, Z., Dekel, T., Cole, F. H., Tucker, R. P., Snavely, N., Liu, C., & Freeman, W. T. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2974454 · Cited by 34
NR-SLAM: Nonrigid Monocular SLAM
Rodríguez, J. J. G., Montiel, J. M. M., & Tardós, J. D. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3422004 · Cited by 27
“First, as in [7], we use a regularizer that penalizes differences in the velocities of close points, that is inspired in the one used in DynamicFusion [8].”
MethodRigidFusion: RGB‐D Scene Reconstruction with Rigidly‐moving Objects
Wong, Y., Li, C., Nießner, M., & Mitra, N. J. (2021) · Computer Graphics Forum · doi:10.1111/cgf.142651 · Cited by 31
ASH: A Modern Framework for Parallel Spatial Hashing in 3D Perception
Dong, W., Lao, Y., Kaess, M., & Koltun, V. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3214347 · Cited by 38
Zhuravlev, A. V., Razevig, V. V., Rogozin, A. A., & Chizh, M. A. (2022) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2022.3217593 · Cited by 27
“The chosen TSDF representation of the surface allows tracking the occluded parts of the target if they were seen before and is expected to facilitate application of deformable shapes tracking methods as many of them also rely on TSDF representation of the surface [22], [23].”
MethodLin, S., Miao, A. J., Lu, J., Yu, S., Chiu, Z.-Y., Richter, F., & Yip, M. C. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160746 · Cited by 34
Transferring Category-Based Functional Grasping Skills by Latent Space Non-Rigid Registration
Rodriguez, D., & Behnke, S. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2826057 · Cited by 32
“[15] use optical flow constraints and thus this approach does not perform well with large deformations or changes in color and illumination.”
MethodLightDB: a DBMS for virtual reality video
Haynes, B. M., Mazumdar, A., Alaghi, A., Bałazińska, M., Ceze, L., & Cheung, A. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3231751.3231768 · Cited by 27
“Several recent projects have explored real-time depth map generation using parallel or custom hardware [31, 49, 4, 46] in the context of cloud-based VR streaming.”
BackgroundSurvey of 3D modeling using depth cameras
Xu, H., Xu, J., & Xu, W. (2019) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2019.09.003 · Cited by 27
Chen, C., Zhu, H. J., Wang, L., & Liu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2961266 · Cited by 31
“Dynamicfusion [21] established amodel under a canonical frame, allowing changes of scene to be mapped to the model by geometric transformation.”
BackgroundHead Reconstruction from Internet Photos
Liang, S. Y., Shapiro, L. G., & Kemelmacher-Shlizerman, I. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_23 · Cited by 26
“Examples include non rigid structure from motion methods [7,8], multiview methods [9,10], dynamic kinect fusion [11], and RGB-D based methods [12,13].”
MethodRobust Non-Rigid Motion Tracking and Surface Reconstruction Using $L_0$ Regularization
Guo, K., Xu, F., Wang, Y., Liu, Y., & Dai, Q. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2688331 · Cited by 26
“Most recently, [48] and [49] perform dynamic reconstruction without initial geometry templates, which is more convenient for data recording.”
InfluentialTextureFusion: High-Quality Texture Acquisition for Real-Time RGB-D Scanning
Lee, J. H., Ha, H., Dong, Y., Tong, X., & Kim, M. H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00135 · Cited by 26
Human-pose estimation based on weak supervision
Hu, X., Bao, X., Wei, G., & Li, Z. (2023) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2022.08.010 · Cited by 29
Depth Map Denoising Network and Lightweight Fusion Network for Enhanced 3D Face Recognition
Xu, R., Wang, K., Deng, C., Wang, M., Chen, X., Huang, W., Deng, W., & Deng, W. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109936 · Cited by 28
“DynamicFusion [13] extends this goal to non-rigid scenes.”
BackgroundVideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams
Wang, L., Yao, K., Guo, C., Zhang, Z., Hu, Q., Jingyi Yu, A., Xu, L., & Wu, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00052 · Cited by 26
“The conventional RGB [8, 24, 31, 37, 38, 50, 80] or RGB-D [10, 20–22, 42, 70, 71] solutions have been extensively investigated.”
BackgroundWang, C., Yuan, J., & Xie, L. (2017) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2017.8023500 · Cited by 32
“Compared with feature-based and direct methods, works like [9]–[11] are able to create dense maps which are crucial for higher level applications, such as object detection and scene understanding.”
MethodVideo to fully automatic 3D hair model
Liang, S. Y., Huang, X., Meng, X., Chen, K., Shapiro, L. G., & Kemelmacher-Shlizerman, I. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275020 · Cited by 26
MOTSLAM: MOT-assisted monocular dynamic SLAM using single-view depth estimation
Zhang, H., Uchiyama, H., Ono, S., & Kawasaki, H. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982280 · Cited by 32
“Another one is to simultaneously track moving objects and perform camera tracking as well as mapping [8], [9], [10], [11], [12], [13], [14], as referred to as dynamic SLAM.”
Chen, H.-Y., Tretschk, E., Stuyck, T., Kadleček, P., Kavan, L., Vouga, E., & Lassner, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01537 · Cited by 25
“DynamicFusion [35] is a model-free, real-time method for reconstructing general scenes from a single RGB-D video.”
MethodCoded Two-Bucket Cameras for Computer Vision
Wei, M., Sarhangnejad, N., Xia, Z., Gusev, N., Katic, N., Genov, R., & Kutulakos, K. N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_4 · Cited by 30
“In the last decade alone, time-of-flight cameras [1,2] have been widely adopted for vision [3] and computational photography tasks [4–7]; event cameras [8] that support asynchronous imaging have led to new vision techniques for high-speed motion analysis [9] and 3D scanning [10]; high-resolution sensors with dual-pixe…”
BackgroundObject registration in semi-cluttered and partial-occluded scenes for augmented reality
Gao, Q. H., Wan, T. R., Tang, W. J., & Chen, L. F. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6905-5 · Cited by 25
“In terms of reconstruction of a dynamic 3D environment and dynamic object registration, the recent advance in Dynamic Fusion [31] has set a benchmark for research in this area.”
BackgroundZhuravlev, A. V., Razevig, V. V., Chizh, M. A., Ge, D., & Boxiong, H. (2020) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2020.3023443 · Cited by 25
“The required human tracking method can be based on such works as [17] and [18], and many more that have recently appeared.”
MethodRobust 3D face modeling and reconstruction from frontal and side images
Jin, H., Wang, X., Zhong, Z., & Hua, J. (2017) · Computer Aided Geometric Design · doi:10.1016/j.cagd.2016.11.001 · Cited by 24
“Newcombe et al. (2015) presented a Kinect based 3D reconstruction method for non-rigid objects such as human face.”
Tele-Aloha: A Telepresence System with Low-budget and High-authenticity Using Sparse RGB Cameras
Tu, H., Shao, R., Dong, X., Zheng, S., Zhang, H., Chen, L., Wang, M., Li, W., Ma, S., Zhang, S., et al. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657491 · Cited by 39
“Theprimitive telepresence systems [Gibbs et al. 1999; Kuster et al. 2012] attempt to capture the appearance and geometry of participants, thus they typically employ depth sensors [Izadi et al. 2011; Newcombe et al. 2015] to serve as a geometry proxy.”
Scene reconstruction techniques for autonomous driving: a review of 3D Gaussian splatting
Zhu, H., Zhang, Z., Zhao, J., Duan, H., Ding, Y., Xiao, X., & Yuan, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10955-4 · Cited by 30
Tracking for mobile devices: A systematic mapping study
Roberto, R. A., Lima, J. P., & Teichrieb, V. (2016) · Computers & Graphics · doi:10.1016/j.cag.2016.02.002 · Cited by 28
Dynamic polygon cloud compression
Pavéz, E., & Chou, P. A. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952694 · Cited by 24
Relightable Neural Human Assets from Multi-view Gradient Illuminations
Zhou, T., He, K., Wu, D., Xu, T. G., Zhang, Q., Shao, K., Chen, W., Xu, L., & Jingyi Yu, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00420 · Cited by 24
“MVS reconstructs 3D geometry from a set of 2D images captured at different viewpoints [26, 29, 74] successfully recovering human body [12, 16, 63], face [9, 42, 51], clothing [28], hair [61], etc.”
BackgroundTemplateless Non-Rigid Reconstruction and Motion Tracking With a Single RGB-D Camera
Wang, K., Zhang, G., & Xia, S. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2740624 · Cited by 24
“Another real-time system, VolumeDeform [22] preserved fine-scale detail in the reconstruction by using a higher-resolution deformation field than the coarse one in DynamicFusion.”
MethodInfluentialTransferring Grasping Skills to Novel Instances by Latent Space Non-Rigid Registration
Rodríguez, D., Cogswell, C., Koo, S., & Behnke, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461169 · Cited by 24
“[19] proposed a non-rigid dense surface reconstruction method using dense non-rigid registration.”
MethodDispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies
Shimada, S., Golyanik, V., Tretschk, E., Stricker, D., & Theobalt, C. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00013 · Cited by 24
“Note that in many related tasks, deformations are parametrised by lower-resolution data structures such as deformation graphs [52, 41, 60].”
BackgroundLearning Residual Flow as Dynamic Motion from Stereo Videos
Lee, S., Im, S., Lin, S. C.-F., & Kweon, I. S. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967970 · Cited by 31
“The ability to infer the motion of cameras and objects enables robots to reason and operate in dynamic real-world, such as autonomous navigation [1], [2], dynamic scene reconstruction [3], and robot manipulation [4].”
BackgroundVariational Level Set Evolution for Non-rigid 3D Reconstruction from a Single Depth Camera
Slavcheva, M., Baust, M., & Ilić, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2976065 · Cited by 24
“Yet, our mean `1 error of 7.7 cm over the entire sequence is not too far from that of other single-stream methods that do not employ priors, namely 4.4 cm for DynamicFusion [18] and 3.7 cm for VolumeDeform [19].”
BackgroundInfluentialRigidity controllable as-rigid-as-possible shape deformation
Chen, S., Gao, L., Lai, Y., & Xia, S. (2017) · Graphical Models · doi:10.1016/j.gmod.2017.02.005 · Cited by 27
“The ARAP energy has also been applied to dynamic shape reconstruction [31, 32].”
MethodFast capture of textured full-body avatar with RGB-D cameras
Lin, S., Chen, Y., Lai, Y., Martin, R. R., & Cheng, Z. (2016) · The Visual Computer · doi:10.1007/s00371-016-1245-9 · Cited by 23
“The differences between our method and the one in [20] should be further examined.”
BackgroundMethodResultInfluentialReal-Time Rendering of Point Clouds With Photorealistic Effects: A Survey
Kivi, P. E. J., Makitalo, M. J., Žádník, J., Ikkala, J., Vadakital, V. K. M., & Jääskeläinen, P. (2022) · IEEE Access · doi:10.1109/access.2022.3146768 · Cited by 23
“Many of the fusion-based systems reconstruct the scenes directly into mesh representations with an intermediate scene description based on signed distance fields (SDF) at capture time [10]– [13].”
BackgroundRobust RGB-D SLAM in dynamic environment using faster R-CNN
Yang, S., Wang, J., Wang, G., Hu, X., Zhou, M., & Liao, Q. (2017) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/compcomm.2017.8322965 · Cited by 23
“Some approaches [11,12,13,14] have been proposed for dynamic environment better in some aspect.”
BackgroundLocal Deformable 3D Reconstruction with Cartan's Connections
Parashar, S., Pizarro, D., & Bartoli, A. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2920821 · Cited by 23
“Other possible ways to recover the depth of deformable objects are by using an active sensor such as Kinect [26], [32], [52], a synchronized system of multiple cameras [9], [19], [42] or machine learning [30], [39], [45], [51].”
BackgroundRana, S., Hampson, R., & Dobie, G. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2946629 · Cited by 23
“Hence, application of this method [24] may not fulfil the requirement of the proposed method.”
MethodLearning to Optimize Non-Rigid Tracking
Li, Y., Božič, A., Zhang, T., Ji, Y., Harada, T., & NieBner, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00496 · Cited by 23
HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences
Tan, F., Tang, D., Dou, M., Guo, K., Pandey, R., Keskin, C., Du, R., Sun, D., Bouaziz, S., Fanello, S., et al. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00186 · Cited by 26
“[45] Richard A Newcombe, Dieter Fox, and Steven M Seitz.”
InfluentialSHARP: Shape-Aware Reconstruction of People in Loose Clothing
Jinka, S. S., Srivastava, A., Pokhariya, C., Sharma, A. K., & Narayanan, P. J. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01736-z · Cited by 27
Lu, Y., Lee, J., Yeh, S.-H., Cheng, H.-M., Chen, B., & Song, D. (2019) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-28619-4_51 · Cited by 23
“Depth sensors such as RGB-D cameras or lidars [12, 7, 21] can help address the scale drifting issue in monocular vision.”
BackgroundToward automatic robotic massage based on interactive trajectory planning and control
Xu, Q., Deng, Z., Zeng, C., Li, Z., He, B., & Zhang, J. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01384-5 · Cited by 22
“A method of dynamic fusion [21] that detects non-rigid deformation reconstructs the input scene into a frame to detect surface deformation.”
GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis
Zhao, B., Li, Y., Sun, Z., Zeng, L., Shen, Y., Ma, R., Zhang, Y., Bao, H., & Cui, Z. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657417 · Cited by 40
Gradient Flow Evolution for 3D Fusion From a Single Depth Sensor
Kang, J., Lee, S., Jang, M., & Lee, S. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3089695 · Cited by 22
Video fields: fusing multiple surveillance videos into a dynamic virtual environment
Du, R., Bista, S., & Varshney, A. (2016) · international conference on 3D web technology · doi:10.1145/2945292.2945299 · Cited by 22
CurveFusion: Reconstructing Thin Structures from RGBD SequencesPreprint
Liu, L., Chen, N., Ceylan, D., Theobalt, C., Wang, W., & Mitra, N. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05284 · Cited by 22
“We would like to extend our method to also capture such dynamic structures borrowing ideas from DynamicFusion [Newcombe et al. 2015] and variations designed for a comparable surface setting.”
MethodDitto in the House: Building Articulation Models of Indoor Scenes through Interactive Perception
Hsu, C.-C., Jiang, Z., & Zhu, Y. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10161431 · Cited by 21
“Recent advances in structure sensors and SLAM algorithms [9, 29, 30] have offered ways to construct static replicas of realworld scenes at unprecedented fidelity.”
BackgroundHUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge
Yoon, J. S., Yu, Z., Park, J., & Park, H. S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3138762 · Cited by 26
“Instead, we represent the dynamic garment shape with per-vertex 3D warping fields [64] that map the garment mesh vertices at the rest pose V to the deformed garment”
BackgroundNeural Density-Distance Fields
Ueda, I., Fukuhara, Y., Kataoka, H., Aizawa, H., Shishido, H., & Kitahara, I. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_4 · Cited by 21
Tracking monocular camera pose and deformation for SLAM inside the human body
Rodríguez, J. J. G., Montiel, J. M. M., & Tardós, J. D. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981203 · Cited by 21
“2) Spatial regularizer: Following [11] we encode assumption (2) with a regularizer that constrains deformations locally so they are spatially smooth: Eti,spa = ∑ j∈G(i) ρ ( ∥wtij(δ t i − δ t j)∥ 2 Σspa ) (8) Here G represents a weighted graph that encodes related points whose deformations should be regularized togethe…”
MethodInfluentialLow-Latency Visual-Based High-Quality 3-D Reconstruction Using Point Cloud Optimization
Chi, P., Wang, Z., Liao, H., Li, T., Zhan, J., Wu, X., Tian, J., & Zhang, Q. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3297232 · Cited by 21
“[32] and Fusion4d proposed by Dou et al.”
BackgroundXiang, D., Prada, F., Cao, Z., Guo, K., Wu, C., Hodgins, J. K., & Bagautdinov, T. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618136 · Cited by 21
“Volumetric fusion [Dou et al. 2017, 2016; Newcombe et al. 2015] and multiview-conditioned implicit functions [Shao et al. 2022a; Suo et al. 2021; Yu et al. 2021b] have been utilized to re-construct scene geometry from the sensor input.”
BackgroundMethodLi, C., Zhao, Z., & Guo, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_20 · Cited by 24
Jiang, Y., Li, J., Qin, H., Dai, Y., Liu, J., Zhang, G., Zhang, C., & Yang, T. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3443637 · Cited by 20
Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences
Müller, N., Wong, Y., Mitra, N. J., Dai, A., & Nießner, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00601 · Cited by 24
“rstanding dynamic environments by object tracking. To achieve general non-rigid object tracking, research focuses on the single object scenario, typically leveraging as-rigid-as-possible registration [34,21, 16,10,13,4]. For multiple object tracking, object rigidity is assumed, and objects are detected and then tracke…”
Physical Interaction: Reconstructing Hand-object Interactions with Physics
Hu, H., Yi, X., Zhang, H., Yong, J., & Xu, F. (2022) · SIGGRAPH Asia Conference Papers · doi:10.1145/3550469.3555421 · Cited by 28
“Similar to [Zhang et al. 2021], we use sphere-mesh [Tkach et al. 2016] to model the hand and Truncated Signed Distance Function (TSDF) [Newcombe et al. 2015] to model the object.”
BackgroundMethodInfluentialScene Representations for Robotic Spatial Perception
Mascaro, R., & Chli, M. (2024) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-040423-030709 · Cited by 34
InterTrack: Tracking Human Object Interaction Without Object Templates
Xie, X., Lenssen, J. E., & Pons‐Moll, G. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00135 · Cited by 29
“In this paper, we address the problem of tracking human object interaction from a single RGB camera, without any object templates.”
Multiframe Scene Flow with Piecewise Rigid Motion
Golyanik, V., Kim, K., Maier, R. L., NieBner, M., Stricker, D., & Kautz, J. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00039 · Cited by 20
Surface Motion Capture Animation Synthesis
Boukhayma, A., & Boyer, E. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2831233 · Cited by 20
“AFTER more than a decade of predominance of standard motion capture, we have witnessed in the recent years the emergence of 4D(3D+t) surface capture either through high quality multi-view set-ups [1], [2], [3], [4], [5] or in a more democratized form with low-cost sensors [6], [7], [8].”
Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors
Lu, Y., Zhao, S., Younes, N., & Hahn, J. K. (2018) · Computer Animation and Virtual Worlds · doi:10.1002/cav.1807 · Cited by 22
“3,21,22 However, reconstruction from continuous deforming inputs requires excessive nonrigid registration and data preprocessing such as temporal denoising.”
BackgroundScalableFusion: High-resolution Mesh-based Real-time 3D Reconstruction
Schreiberhuber, S., Prankl, J., Patten, T., & Vincze, M. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793654 · Cited by 20
“The most common approaches are based on KinectFusion ([7], [10], [15]) and ElasticFusion ([16], [17], [8]).”
BackgroundMethodSame Features, Different Day: Weakly Supervised Feature Learning for Seasonal Invariance
Spencer, J., Bowden, R., & Hadfield, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00649 · Cited by 25
Towards Precise Completion of Deformable Shapes
Halimi, O., Imanuel, I., Litany, O., Trappolini, G., Rodolà, E., Guibas, L. J., & Kimmel, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58586-0_22 · Cited by 24
“The methods that can handle substantial partiality usually rely on some modification of the iterative closest point (ICP) algorithm and often have dif-ficulties in handling large deformations between the full and the partial shape [66,50].”
MethodHigh-Speed Dynamic Projection Mapping onto Human Arm with Realistic Skin Deformation
Peng, H.-L., & Watanabe, Y. (2021) · Applied Sciences · doi:10.3390/app11093753 · Cited by 21
SLAMM: Visual monocular SLAM with continuous mapping using multiple maps
Daoud, H. A., Sabri, A. Q. M., Loo, C. K., & Alsahag, A. M. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195878 · Cited by 24
Dynamic human body reconstruction and motion tracking with low-cost depth cameras
Wang, K., Zhang, G., Yang, J. J., & Bao, H. (2020) · The Visual Computer · doi:10.1007/s00371-020-01826-4 · Cited by 20
“DynamicFusion [8] and VolumeDeform [46] were designed to reconstruct nonrigidly deforming scenes in real time and were applied for reconstruction of a moving person.”
MethodSMASH: physics-guided reconstruction of collisions from videos
Monszpart, Á., Thuerey, N., & Mitra, N. J. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982421 · Cited by 19
“In case of dynamic scenes, even state-of-the-art RGBD based systems [Newcombe et al. 2015; Dou et al. 2015] struggle with highly dynamic motion and occlusion.”
NerfCap: Human Performance Capture With Dynamic Neural Radiance Fields
Wang, K., Peng, S., Zhou, X., Yang, J. J., & Zhang, G. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3202503 · Cited by 23
“2.1 Template-Free Human Performance Capture Template-free methods [6], [7] based on depth sensors obtain impressive human reconstruction in realtime by volumetri-cally fusing captured depth maps, but they only works reliably for small and slow motions.”
BackgroundMethodInfluentialWoo, T., Park, W., Jeong, W., & Park, J. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.09.013 · Cited by 22
In-process 4D reconstruction in robotic additive manufacturing
Chew, S. Y., Asadi, E., Vargas-Uscategui, A., King, P. C., Gautam, S., Bab‐Hadiashar, A., & Cole, I. S. (2024) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2024.102784 · Cited by 22
ISNN: Impact Sound Neural Network for Audio-Visual Object Classification
Sterling, A., Wilson, J., Lowe, S., & Lin, M. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_34 · Cited by 21
“RGB-D depth-based, active reconstruction methods can also be used to generate 3D geometrical models of static [18,6] and dynamic [19,20] scenes using commodity sensors such as the Microsoft Kinect and GPU hardware in real-time.”
BackgroundGlobal 3D Non-Rigid Registration of Deformable Objects Using a Single RGB-D Camera
Yang, J., Guo, D., Li, K., Wu, Z., & Lai, Y. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2909197 · Cited by 19
TwinFusion: High Framerate Non-rigid Fusion through Fast Correspondence Tracking
Guo, K., Taylor, J., Fanello, S., Tagliasacchi, A., Dou, M., Davidson, P., Kowdle, A., & Izadi, S. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00074 · Cited by 21
“Digitizing generic non-rigid geometry General nonrigid reconstruction techniques such as [23, 16, 7] lack a fixed model, so incrementally accumulating a model through frame-to-frame tracking remains the only viable option.”
BackgroundMethodInfluentialRobust 3D Human Motion Reconstruction via Dynamic Template Construction
Li, Z., Ji, Y., Yang, W. H., Ye, J., & Jingyi Yu, A. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00063 · Cited by 21
Marching-Primitives: Shape Abstraction from Signed Distance Function
Liu, W., Wu, Y., Ruan, S., & Chirikjian, G. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00847 · Cited by 38
“It is the basic framework for many classic 3D reconstruction algorithms such as TSDF volume reconstruction [10, 15], KinectFusion [19], and DynamicFusion [26].”
BackgroundMethodClothed Human Performance Capture with a Double-layer Neural Radiance Fields
Wang, K., Zhang, G., Cong, S., & Yang, J. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02021 · Cited by 18
“Previous systems [4, 21, 38, 39] reconstruct 3D clothed humans by using depth sensors or fitting a personalized template [9, 10, 37] to the image observations (e.g., body joints and silhouettes).”
BackgroundMethodDynamic Non-Rigid Objects Reconstruction with a Single RGB-D Sensor
Wang, S., Zuo, X., Du, C., Wang, R., Zheng, J., & Yang, R. (2018) · Sensors · doi:10.3390/s18030886 · Cited by 19
“As compared to some previous dynamic fusion works [16–18] that suffer from the error accumulation problem, the main contribution of this paper is that we address this drifting problem by enforcing the loop closure constraints explicitly.”
MethodResultLacher, R., Vasconcelos, F., Williams, N. R., Rindermann, G., Hipwell, J. H., Hawkes, D. J., & Stoyanov, D. V. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.01.003 · Cited by 18
3D Visual passcode: Speech-driven 3D facial dynamics for behaviometrics
Zhang, J., & Fisher, R. B. (2019) · Signal Processing · doi:10.1016/j.sigpro.2019.02.025 · Cited by 24
“However, we preserve the raw sequences with 500 fps in the public dataset, since the extra frames are also helpful in generating improved 3D data by fusion-based spatio-temporal noise reduction algorithms [56, 57].”
MethodEfficient 3D Reconstruction and Streaming for Group-Scale Multi-Client Live Telepresence
Stotko, P., Krumpen, S., Weinmann, M., & Klein, R. (2019) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2019.00018 · Cited by 18
“This approach has been well-established by previous investigations in the context of real-time reconstruction [1, 2, 9, 11–13,21–23,30, 31] and telepresence [20, 24, 27].”
ResultSimultaneous Tracking and Elasticity Parameter Estimation of Deformable Objects
Sengupta, A., Lagneau, R., Krupa, A., Marchand, É., & Marchal, M. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196770 · Cited by 18
“Both physics-based model such as [1], [2] or geometric-based models such as [3], [4] offer efficient mechanisms to track complex 3D shapes without relying on any kind of training data.”
BackgroundiButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering
Wang, L., Wang, Z., Lin, P., Jiang, Y., Suo, X., Wu, M., Xu, L., & Yu, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475412 · Cited by 26
“With the utility of the depth sensor and the powerful computing power of GPU, some works [15, 31, 40, 47, 54] combine the volumetric fusion and non-rigid tracking to reconstruct free-form human body in real-time by using only a single RGBD sensor.”
Local to global registration of multi-view range scans using spanning tree
Zhu, J., Zhu, L., Jiang, Z., Bai, X., Li, Z., & Wang, L. (2016) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2016.07.003 · Cited by 18
Reconstructing High-Resolution Face Models From Kinect Depth Sequences
Bondi, E., Pala, P., Berretti, S., & Bimbo, A. D. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2601059 · Cited by 18
“This completely disregards the temporal dimension available in depth sequences acquired with a Kinect sensor.”
BackgroundFusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras
Meerits, S., Thomas, D., Nozick, V., & Saitô, H. (2018) · Computational Visual Media · doi:10.1007/s41095-018-0121-0 · Cited by 18
“[6] introduced DynamicFusion, wherein depth data is accumulated into a canonical model of a scene, which is subsequently deformed using a warp field to match scene changes in real time.”
BackgroundDynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics ModelingPreprint
Zhang, M., Zhang, K., & Li, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.18912 · Cited by 48
“Our approach is more related to another line of work, which focuses on lifting RGB or RGB-D images to 3D and track in the 3D space [14–16].”
BackgroundShared Realities: Avatar Identification and Privacy Concerns in Reconstructed Experiences
Wang, C. Y., Sriram, S., & Won, A. S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476078 · Cited by 18
“Accurate methods based on stereo or fusion have been proposed using various types of sensors [11, 54, 65, 71], and several applications have become popular in sports, medicine and entertainment (e.”
MethodDynamic Object-Aware Visual Odometry (VO) Estimation Based on Optical Flow Matching
Cho, H. M., & Kim, E. (2023) · IEEE Access · doi:10.1109/access.2023.3241961 · Cited by 19
“Before deep learning was developed, many studies attempted to use human intervention to perform robust odometry estimation or SLAM in dynamic environments [22], [23].”
Metadata
- Authors
- Newcombe, R. A., Fox, D., & Seitz, S. M.
- Affiliations
- University of Washington (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 343-352
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1938204631
- Cited by
- 951
- References
- 42
- Citations by year
- 2026: 28 · 2025: 61 · 2024: 67 · 2023: 93 · 2022: 84 · 2021: 113 · 2020: 105 · 2019: 124 · 2018: 110 · 2017: 78 · 2016: 80 · 2015: 8
- FWCI
- 159.57
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- dense SLAM, real-time 3D reconstruction, KinectFusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.