KinectFusion: Real-time dense surface mapping and tracking
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103 · cited 2×
“The most important class of algorithms have been based on the ICP concept introduced in [3] which poses data alignment as a nonlinear optimisation problem in which correspondences between scans are approximated using the closest pairs of points found between scans at the previous iteration.”
BackgroundMethodMarching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104 · cited 1×
“One option is to extract the connected surfaces using a marching cubes type algorithm [18], followed by a standard rasterising rendering pipeline.”
MethodBilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“We apply a bilateral filter [27] to the raw depth map to obtain a discontinuity preserved depth map with reduced noise Dk,”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 3×
“By using only depth data, our system can work in complete darkness mitigating any issues concerning low light conditions, problematic for passive camera [17, 19, 26] and RGB-D based systems [14].”
BackgroundMethodObject modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 6×
“A related non-parametric representation used in graphics is the signed distance function (SDF) introduced in [7] for the purpose of fusing partial depth scans while mitigating problems related to mesh-based reconstruction algorithms.”
BackgroundMethodInfluentialA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“In computer vision, research on structure from motion (SFM) and multi-view stereo (MVS) has produced many compelling re-sults, in particular accurate camera tracking and sparse reconstructions (e.g. [10]), and increasingly reconstruction of dense surfaces (e.g. [24]).”
BackgroundIterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,402 · cited 1×
“A number of ICP variants perform early iterations on a subset of possibly corresponding points or operate within a coarse-to-fine scheme [31], speeding up both the data association and final pose optimisation.”
BackgroundKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 1×
“In [16] we discuss all these possibilities in detail.”
BackgroundSelf-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
Self-citation
Research works printed but not linked: 4
- weise, 0, Inhand scanning with online loop closure, Proceedings of the IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM) 2009
- wurm, 0, OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems, Proceedings of the ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation 2010
- 0, NVIDIA OptiX 2 Ray Tracing Engine
- zach, 0, A globally optimal algorithm for robust TV-L1 range image integration, Proceedings of the International Conference on Computer Vision (ICCV) 2007
Cited by
Items 501–600 of 1,006
Learning to Solve Nonlinear Least Squares for Monocular Stereo
Clark, R., Bloesch, M., Czarnowski, J., Leutenegger, S., & Davison, A. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_18 · Cited by 92
Self-citation3D-aware Image Generation using 2D Diffusion Models
Xiang, J., Yang, J., Huang, B., & Tong, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00226 · Cited by 92
SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans
Avetisyan, A., Khanova, T., Choy, C., Dash, D. H., Dai, A., & Nießner, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_36 · Cited by 91
Survey and Evaluation of RGB-D SLAM
Zhang, S., Zheng, L., & Tao, W. (2021) · IEEE Access · doi:10.1109/access.2021.3053188 · Cited by 91
“Architecture of KinectFusion [12]. the tracking module, it is necessary to perform feature extraction operations on each frame.”
MethodInfluentialLi, L., Wang, R., & Zhang, X. (2021) · Mathematical Problems in Engineering · doi:10.1155/2021/9953910 · Cited by 91
“Meanwhile, a point cloud is often combined with a traditional camera [69] to effectively address real-world problems such as object tracking, object recognition, and object detection.”
BackgroundQuality-driven poisson-guided autoscanning
Wu, S., Sun, W., Long, P., Huang, H. F., Cohen‐Or, D., Gong, M., Deußen, O., & Chen, B. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661242 · Cited by 91
“KinectFusion based systems [Newcombe et al. 2011; Chen et al. 2013] recently gained popularity for real-time dense surface mapping, tracking, and reconstruction with mobile low-cost depth cameras.”
BackgroundA real-time method for depth enhanced visual odometry
Zhang, J., Kaess, M., & Singh, S. (2015) · Autonomous Robots · doi:10.1007/s10514-015-9525-1 · Cited by 91
“With a single camera (Klein and Murray 2007; Newcombe et al. 2011; Engel et al. 2013; Forster et al. 2014), if assuming unconstrained motion, rotation can be recovered but translation is up to scale.”
BackgroundA Kinect-Based Approach for 3D Pavement Surface Reconstruction and Cracking Recognition
Zhang, Y., Chen, C., Wu, Q., Lu, Q., Zhang, S., Zhang, G., & Yang, Y. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2791476 · Cited by 91
“The constructed pavement mesh contains 3D geometry information of the pavement surface (we refer readers to related studies [52], [56] for a more detailed explanations of algorithmic procedures of Kinect fusion).”
BackgroundSemantic mapping for orchard environments by merging two‐sides reconstructions of tree rows
Dong, W., Roy, P., & Isler, V. (2019) · Journal of Field Robotics · doi:10.1002/rob.21876 · Cited by 91
ManifoldPlus: A Robust and Scalable Watertight Manifold Surface Generation Method for Triangle SoupsPreprint
Huang, J., Zhou, Y., & Guibas, L. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11621 · Cited by 91
“2006; Kazhdan and Hoppe 2013], scanning sensor location [Curless and Levoy 1996; Newcombe et al. 2011], or geometry heuristics [Boissonnat 1984; Hornung and Kobbelt 2006].”
BackgroundiMoon: Using Smartphones for Image-based Indoor Navigation
Dong, J., Xiao, Y., Noreikis, M., Ou, Z., & Ylä-Jääski, A. (2015) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/2809695.2809722 · Cited by 90
Wei, Y., & Akinci, B. (2019) · Automation in Construction · doi:10.1016/j.autcon.2019.102915 · Cited by 90
Autonomous Flights in Dynamic Environments with Onboard Vision
Wang, Y., Ji, J., Wang, Q., Xu, C., & Gao, F. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636117 · Cited by 90
“Kinect Fusion [15] implements Iterative Closest Point (ICP) [16] to deal with moving objects.”
MethodPINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence
Dong, Z., Guo, C., Song, J., Chen, X., Geiger, A., & Hilliges, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01982 · Cited by 91
Self-citationSQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation
Wang, Y., Liang, Y., Xu, H., Jiao, S., & Yu, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28383 · Cited by 90
López-Guillén, E., García, S., Barea, R., Bergasa, L. M., Molinos, E. J., Arroyo, R., Romera, E., & Pardo, S. (2017) · Sensors · doi:10.3390/s17040802 · Cited by 89
“For this reason, we use low-cost commercial platforms and an open-source development environment (ROS), so that drivers of sensors and some algorithms can be used without”
MethodReal-Time Progressive 3D Semantic Segmentation for Indoor Scenes
Pham, Q.-H., Hua, B., Nguyen, T. M., & Yeung, S.-K. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00121 · Cited by 90
SLAM with objects using a nonparametric pose graph
Mu, B., Liu, S.-Y., Paull, L., Leonard, J. J., & How, J. P. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759677 · Cited by 89
“This simplified representation of the world facilitates efficient computation, and thus real-time performance can be achieved on relatively large scenes with a single CPU.”
BackgroundMobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone
Yang, X., Zhou, L., Jiang, H., Tang, Z.-L., Wang, Y., Bao, H., & Zhang, G. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3023634 · Cited by 89
“TSDF fusion has demonstrated its effectiveness in literature [1, 24].”
BackgroundMethodInfluential3D Surveying of Underground Built Heritage: Opportunities and Challenges of Mobile Technologies
Stefano, F. D., Torresani, A., Farella, E. M., Pierdicca, R., Menna, F., & Fabio Remondino, A. (2021) · Sustainability · doi:10.3390/su132313289 · Cited by 89
Probabilistic object tracking using a range camera
Wuthrich, M., Pastor, P., Kalakrishnan, M., Bohg, J., & Schaal, S. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696810 · Cited by 79
“It has been shown that using depth data can add robustness to tasks such as SLAM [13], object detection and recognition [12, 9].”
MethodFLARE: Fast layout for augmented reality applications
Gal, R., Shapira, L., Ofek, E., & Kohli, P. (2014) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2014.6948429 · Cited by 89
“Index Terms: F.4.1 [Mathematical Logic]: Logic and Constraint Programming—; G.3 [Probability and Statistics]: Markov Processes—;”
BackgroundInfluentialInferring Forces and Learning Human Utilities from Videos
Zhu, Y., Jiang, C., Zhao, Y., Terzopoulos, D., & Zhu, S.-C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.415 · Cited by 79
“Reconstructing closed-loop scenes: Reconstruction methods that use purely geometric registration [48, 34, 49, 65] suffer from aliasing of fine geometric details and an inability to disambiguate different locations based on local geometry.”
BackgroundGlobal Pose Estimation with an Attention-Based Recurrent Network
Parisotto, E., Chaplot, D. S., Zhang, J., & Salakhutdinov, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00061 · Cited by 88
ASRO-DIO: Active Subspace Random Optimization Based Depth Inertial Odometry
Zhang, J., Tang, Y., Wang, H., & Xu, K. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3208503 · Cited by 88
“We adopt the truncated signed distance field (TSDF) based mapping framework [25], [26] which allows for depth-based fusion.”
MethodDiffComplete: Diffusion-based Generative 3D Shape Completion
Chu, R., Xie, E., Mo, S., Li, Z., Nießner, M., Fu, C., & Jia, J. (2023) · neural information processing systems · doi:10.52202/075280-3318 · Cited by 88
UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction Using Commercial RGBD Cameras
Xu, L., Su, Z., Han, L., Yu, T., Liu, Y., & Fang, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2915229 · Cited by 88
“Only recently, free-form dynamic reconstruction meth-ods with real-time performance have been proposed by combining the volumetric fusion [40], [41] and the nonrigid tracking techniques.”
MethodContinuous humanoid locomotion over uneven terrain using stereo fusion
Fallon, M. F., Marion, P., Deits, R., Whelan, T. J., Antone, M. E., McDonald, J., & Tedrake, R. (2015) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2015.7363465 · Cited by 87
Good Feature Matching: Toward Accurate, Robust VO/VSLAM With Low Latency
Zhao, Y., & Vela, P. A. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2964138 · Cited by 87
“Efforts have focused on the accuracy and robustness of pose tracking [1]–[6] and mapping [7], [8], while meeting the real-time requirement (e.”
BackgroundMahajan, H. B., Uke, N., Pise, P. D., Shahade, M., Dixit, V. G., Bhavsar, S., & Deshpande, S. D. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14253-5 · Cited by 87
“Various systems, such as MonoSLAM [66], Parallel Tracking And Mapping (PTAM) [28], Fusion Algorithm Based Mapping (FAB-MAP) [16], Dense Tracking And Mapping (DTAM) [52], and KinectFusion [53], have been introduced.”
BackgroundMethodInfluentialImproved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications
Yeung, K., Kwok, T.-H., & Wang, C. C. L. (2013) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4025404 · Cited by 87
“Specifically, 3D point clouds are used to calibrate the positions of two Kinects by a method similar to [20] so that the 3D environments captured by KA and KB overlap.”
BackgroundMethodToward lifelong object segmentation from change detection in dense RGB-D maps
Finman, R., Whelan, T. J., Kaess, M., & Leonard, J. J. (2013) · European Conference on Mobile Robots · doi:10.1109/ecmr.2013.6698839 · Cited by 87
“D object discovery method is two RGBD maps that have some region of overlap where the map may have changed.”
MethodOleynikova, H., Lanegger, C., Taylor, Z., Pantic, M., Millane, A., Siegwart, R., & Nieto, J. I. (2020) · Journal of Field Robotics · doi:10.1002/rob.21950 · Cited by 87
“We use two different types of SDFs: Truncated Signed Distance Fields (TSDFs), based on Curless and Levoy [19] and KinectFusion [20] for integrating point cloud data, in a method that gives a more accurate surface estimate than occupancybased methods, but uses projective distances and truncates the value to a small ban…”
MethodMapGENIE: Grammar-enhanced indoor map construction from crowd-sourced data
Philipp, D., Baier, P., Dibak, C., Dürr, F., Rothermel, K., Becker, S., Peter, M., & Fritsch, D. (2014) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom.2014.6813954 · Cited by 86
“Recently, also human-operated SLAM approaches have emerged which employ miniature laser scanners [13], or the Microsoft Kinect [14].”
BackgroundRecchiuto, C. T., & Sgorbissa, A. (2017) · Journal of Field Robotics · doi:10.1002/rob.21756 · Cited by 86
DEVO: Depth-Event Camera Visual Odometry in Challenging Conditions
Zuo, Y., Yang, J., Chen, J., Wang, X., Wang, Y., & Kneip, L. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811805 · Cited by 86
Real-time photometric registration from arbitrary geometry
Gruber, L., Richter-Trummer, T., & Schmalstieg, D. (2012) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2012.6402548 · Cited by 86
“Recent results in online reconstruction [13] have shown that geometry acquisition can be achieved without tedious preparations, which has important implications concerning the feasibility of high-quality AR.”
BackgroundMethodInfluentialHole Filling With Multiple Reference Views in DIBR View Synthesis
Li, S., Zhu, C., & Sun, M.-T. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2791810 · Cited by 86
“sis [5], [6] is one of the representative view synthesis methods [7]–[10] due to its capability in rendering virtual views at arbitrary viewpoints.”
BackgroundAutomatic Non-Destructive Growth Measurement of Leafy Vegetables Based on Kinect
Hu, Y., Wang, L., Xiang, L., Wu, Q., & Jiang, H. (2018) · Sensors · doi:10.3390/s18030806 · Cited by 86
“Although some robust modeling algorithms have been proposed, such as KinectFusion [35,36], the results of plant 3D reconstruction are not always satisfactory [37].”
MethodZou, C., Su, J.-W., Peng, C., Colburn, A., Shan, Q., Wonka, P., Chu, H., & Hoiem, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01426-8 · Cited by 85
“2016), and scene reconstructions from point clouds (Newcombe et al. 2011; Monszpart et al. 2015; Liu et al. 2018; Cabral et al. 2014).”
BackgroundMarshall, J. A., Sun, W., & L’Afflitto, A. (2021) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2021.10.013 · Cited by 85
“The other one is the direct approach , which uses all the pixels from a camera frame to resolve the surrounding environment and produces a dense reconstruction map (Newcombe et al., 2011) or semi-dense map (Engel, Schops, & Cremers, 2014).”
BackgroundMethodShapeMap 3-D: Efficient shape mapping through dense touch and vision
Suresh, S., Si, Z., Mangelson, J. G., Yuan, W., & Kaess, M. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812040 · Cited by 85
“Vision and depth-based 3-D perception is well-studied [1], but can often fail in the context of manipulation.”
BackgroundVoxfield: Non-Projective Signed Distance Fields for Online Planning and 3D Reconstruction
Pan, Y., Kompis, Y., Bartolomei, L., Mascaro, R., Stachniss, C., & Chli, M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981318 · Cited by 85
“When it comes to path planning, however, ESDF maps are more informative than occupancy grids or TSDFs as they allow for efficient queries of distances and gradients to obstacles at any point in space.”
BackgroundMethodInfluentialDense real-time mapping of object-class semantics from RGB-D video
Stückler, J., Waldvogel, B., Schulz, H., & Behnke, S. (2013) · Journal of Real-Time Image Processing · doi:10.1007/s11554-013-0379-5 · Cited by 85
“A prominent example is KinectFusion [12] which aligns depth images on the GPU toward a map that is incrementally accumulated from the registered images.”
BackgroundOngoing Evolution of Visual SLAM from Geometry to Deep Learning: Challenges and Opportunities
Li, R., Wang, S., & Gu, D. (2018) · Cognitive Computation · doi:10.1007/s12559-018-9591-8 · Cited by 85
“[28] proposed KinectFusion using a RGB-D camera which is successfully demonstrated in dense registration and mapping.”
MethodJiang, G., Yin, L., Jin, S., Tian, C., Ma, X., & Ou, Y. (2019) · Applied Sciences · doi:10.3390/app9102105 · Cited by 85
“Reference [14] proposes Kinect fusion method, which uses the depth images acquired by Kinect to measure the minimum distance of each pixel in each frame, and fuses all the depth images to obtain global map information.”
MethodTANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view StereoPreprint
Koestler, L., Yang, N., Zeller, N., & Cremers, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07418 · Cited by 85
“While most previous methods optimize only for the frame pose, Schöps et al. [33] propose a full bundle adjustment based direct RGB-D SLAM which optimizes for both camera pose and 3D structure.”
BackgroundMethodInfluentialRandom forests versus Neural Networks — What's best for camera localization?
Massiceti, D., Krull, A., Brachmann, E., Rother, C., & Torr, P. H. S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989598 · Cited by 73
“We use the 7-Scenes dataset [1] which contains RGB-D images captured with a handheld Kinect camera (640× 480 resolution) and associated ground-truth camera pose calculated using the Kinect Fusion implementation of [26].”
MethodGeometric Correspondence Network for Camera Motion Estimation
Tang, J., Folkesson, J. B., & Jensfelt, P. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2794624 · Cited by 84
“Comparison between different structures using KITTI Raw [44] road scenes, we used sequences [1, 2, 5, 9] for training and [11, 13] for testing in this test.”
MethodSurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction
Gao, W., & Tedrake, R. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.029 · Cited by 84
“To address this issue, KinectFusion [13] and many related approaches [22, 21, 3, 10] estimate the camera pose and fuse depth frames online for a complete, smoothed 3D geometry scanning.”
MethodDDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs
Yan, S., Wu, C., Wang, L., Xu, F., An, L., Guo, K., & Liu, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_10 · Cited by 84
“KinectFusion [27] is the first method to show real-time hand-held scanning of large scenes with a consumer depth sensor.”
MethodVisionBlender: a tool to efficiently generate computer vision datasets for robotic surgery
Cartucho, J., Tukra, S., Li, Y., Elson, D. S., & Giannarou, S. (2020) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2020.1835546 · Cited by 85
“A suitable solu-tion for merging multiple depth maps of a static tissue is KinectFusion (Newcombe et al. 2011), and for deforming tissue scenes, SurfelWarp (Gao and Tedrake 2018) can be used.”
MethodLearning to be a depth camera for close-range human capture and interaction
Fanello, S., Keskin, C., Izadi, S., Kohli, P., Kim, D., Sweeney, D., Criminisi, A., Shotton, J., Kang, S. B., & Paek, T. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601223 · Cited by 83
Self-citationSceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields
Cao, A.-Q., & de Charette, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00861 · Cited by 84
A non-rigid map fusion-based direct SLAM method for endoscopic capsule robots
Turan, M., Almalioglu, Y., Araujo, H., Konukoğlu, E., & Sitti, M. (2017) · International Journal of Intelligent Robotics and Applications · doi:10.1007/s41315-017-0036-4 · Cited by 83
“Inspired from large-scale RGB Depth SLAM approaches (Whelan et al. 2015; Newcombe et al. 2011), the proposed method is to the best of our knowledge the first fully dense, direct medical SLAM approach using GPU accelerated nonrigid frame-to-model fusion, joint volumetric-photometric pose estimation and dense model-to-m…”
MethodOctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05009 · Cited by 25
“ese cases, the input to the network is of two-dimensional nature, e.g., in image classification [18], object detection [35] or semantic segmentation [13]. However, recent advances in 3D reconstruction [33] and graphics [21] allow capturing and modeling large amounts of 3D data. At the same time, large 3D repositories s…”
BackgroundClinical paradigms and challenges in surface guided radiation therapy: Where do we go from here?
Batista, V., Meyer, J., Kugele, M., & Al‐Hallaq, H. (2020) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2020.09.041 · Cited by 82
Rapid avatar capture and simulation using commodity depth sensors
Shapiro, A., Feng, A., Wang, R., Li, H., Bolas, M. T., Medioni, G. G., & Suma, E. A. (2014) · Computer Animation and Virtual Worlds · doi:10.1002/cav.1579 · Cited by 82
“Each pose takes 10 seconds and approximately 200 frames are merged using the KinectFusion algorithm [9] (Fig 3(a))).”
MethodEvaluation of Kinect 3D Sensor for Healthcare Imaging
Pöhlmann, S. T. L., Harkness, E. F., Taylor, C., & Astley, S. (2016) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-016-0184-2 · Cited by 82
“’s [9] algorithm is for example available as KinectFusion implemented in the Development Kit.”
MethodDepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition
Planche, B., Wu, Z., Ma, K., Sun, S., Kluckner, S., Lehmann, O., Chen, T., Hutter, A., Zakharov, S., Kosch, H., et al. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00011 · Cited by 83
Dynamic Reconstruction of Deformable Soft-Tissue With Stereo Scope in Minimal Invasive Surgery
Song, J., Wang, J., Zhao, L., Huang, S., & Dissanayake, G. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2735487 · Cited by 81
“Previous works adopt TSDF volume to store and fuse models [9]–[12].”
BackgroundMethodAutomated wall‐climbing robot for concrete construction inspection
Yang, L., Li, B., Feng, J., Yang, G., Yong, C., Jiang, B., & Xiao, J. (2022) · Journal of Field Robotics · doi:10.1002/rob.22119 · Cited by 81
“D frame, given the camera intrinsic parameters K (Zhang, 2000), this paper follows the following steps that inspired by Newcombe et al. (2011) to perform spatial fusion: (1) Warp each voxel ϑ to current frame view given the camera pose R t( , )o o , to obtain the association from 3D voxel cell to…”
MethodFinding planes in LiDAR point clouds for real-time registration
Grant, W. S., Voorhies, R. C., & Itti, L. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696980 · Cited by 81
“There has also been considerable effort to speed up ICP-based approaches through hardware optimization, such as GPGPU acceleration [10] [11] [12], multithreading [13], and sophisticated data structures [14].”
BackgroundFlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras
Xu, L., Liu, Y., Cheng, W., Guo, K., Zhou, G., Dai, Q., & Fang, L. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2728660 · Cited by 81
Live Semantic 3D Perception for Immersive Augmented Reality
Han, L., Zheng, T., Zhu, Y., Xu, L., & Fang, L. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.2973477 · Cited by 81
“With the popularity of RGBD sensors, various approaches [12, 15, 30, 46] have demonstrated significant potentials for live 3D geometry reconstruction of the environment.”
BackgroundInside-Out Instrument Tracking for Surgical Navigation in Augmented Reality
Gsaxner, C., Li, J., Pepe, A., Schmalstieg, D., & Egger, J. (2021) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3489849.3489863 · Cited by 80
“We calibrate the transformation of D to the reference camera L using the same principles as described in Section 3.1 and subsequently use SLAM to localize D in the world space W .”
MethodInfluentialMa, X., Smart, B., Bhalgat, Y. S., Chen, S., Li, X., Ding, J., Gu, J., Chen, D. Z., Peng, S., Bian, J.-W., et al. (2024) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.2405.10255 · Cited by 85
“Each voxel minimally encodes the occupancy information (binary or probabilistically) but can additionally encode distance to a surface, as in the Signed Distance Function (SDF) [53, 54, 55, 56, 57] or Truncated Signed Distance Function (TSDF) [58, 59, 60, 61, 62, 63].”
BackgroundInterest Point Detector and Feature Descriptor Survey
Krig, S. (2014) · Apress eBooks · doi:10.1007/978-1-4302-5930-5_6 · Cited by 80
Semantic alignment of LiDAR data at city scale
Yu, F., Xiao, J., & Funkhouser, T. A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298781 · Cited by 80
“Our experimental results indicate that the proposed methods can achieve significantly better alignments than those provided by alternative methods for four different cities.”
MethodPauwels, K., Rubio, L., Díaz, J., & Ros, E. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.304 · Cited by 80
“Recently, due to the prevalence of cheap depth sensors, depth information is being applied with great success in various related problems, such as on-line scene modeling [15], articulated body pose estimation [18], and texture-less object detection [7].”
BackgroundPose Depth Volume extraction from RGB-D streams for frontal gait recognition
Chattopadhyay, P., Roy, A., Sural, S., & Mukhopadhyay, J. (2013) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.02.010 · Cited by 80
Winkler, A., Havoutis, I., Bazeille, S., Ortiz, J., Focchi, M., Dillmann, R., Caldwell, D. G., & Semini, C. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907815 · Cited by 80
“We use a real map of the environment that is generated offline using a Microsoft Kinect sensor and the KinectFusion [17] algorithm (Fig.”
MethodLitany, O., Rodolà, E., Bronstein, A. M., Bronstein, M. M., & Cremers, D. (2016) · Computer Graphics Forum · doi:10.1111/cgf.12970 · Cited by 80
Multi-Modal Geometric Learning for Grasping and Manipulation
Watkins-Valls, D., Varley, J., & Allen, P. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794233 · Cited by 80
“There has been successful research in utilizing continuous streams of visual information similar to Kinect Fusion [16] or SLAM [17] in order to improve models of 3D objects for manipulation, an example being [18][19].”
MethodSequential Adversarial Learning for Self-Supervised Deep Visual Odometry
Li, S., Xue, F., Wang, X., Yan, Z., & Zha, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00294 · Cited by 80
“The ability for an agent to understand 3D environment and infer ego-motion is crucial for many real-world applications, such as autonomous driving [7], robotics [14], and virtual/augmented reality [30].”
BackgroundStructured Output Learning with Conditional Generative Flows
Lu, Y. L., & Huang, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5940 · Cited by 80
Autoscanning for coupled scene reconstruction and proactive object analysis
Xu, K., Huang, H. F., Shi, Y., Li, H., Long, P., Caichen, J., Sun, W., & Chen, B. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818075 · Cited by 79
Learning 3D Object Categories by Looking Around Them
Novotný, D., Larlus, D., & Vedaldi, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.558 · Cited by 79
“We extract this information using off-theshelf methods: the structure-from-motion (SFM) algorithm COLMAP for RGB sequences [33, 34], and an open-source implementation [31] of KinectFusion (KF) [24] for RGBD sequences.”
MethodReal-Time Dense Reconstruction of Tissue Surface From Stereo Optical Video
Zhou, H., & Jayender, J. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2927436 · Cited by 79
NeuralFusion: Online Depth Fusion in Latent Space
Weder, S., Schönberger, J. L., Pollefeys, M., & Oswald, M. R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00318 · Cited by 79
“This approach especially became popular with the wide availability of lowcost depth sensors like the Kinect and has led to works like KinectFusion [40] and related works like sparse-sequence fusion [66], BundleFusion [12], or variants on sparse grids like VoxelHashing [42], InfiniTAM [46], Voxgraph [47], octree-based…”
Method3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes
Held, J., Vandeghen, R., Hamdi, A., Deliège, A., Cioppa, A., Giancola, S., Vedaldi, A., Ghanem, B., & Droogenbroeck, M. V. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01990 · Cited by 81
“Reconstructing complex scenes and synthesizing novel views have been fundamental challenges in computer vision and graphics for decades [1, 12], with applications ranging from virtual reality to autonomous navigation [17, 36, 40].”
Fine-to-Coarse Global Registration of RGB-D Scans
Halber, M., & Funkhouser, T. A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.705 · Cited by 79
“More recent methods align frames to a scene model, represented as a point cloud [19, 21, 34, 46] or an implicit function [6, 9, 20, 28, 42, 44, 45].”
BackgroundMethodGeometrically consistent plane extraction for dense indoor 3D maps segmentation
Pham, T. T., Eich, M., Reid, I. D., & Wyeth, G. F. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759618 · Cited by 79
“Specifically, we select 200 frames around each ground truth labelled image to reconstruct the scene using efficient InfiniTAM [26], which is an open source implementation of Kinect-Fusion [27].”
MethodSLAMBench2: Multi-Objective Head-to-Head Benchmarking for Visual SLAM
Bodin, B., Wagstaff, H., Saecdi, S., Nardi, L., Vespa, E., Mawer, J., Nisbet, A. P., Luján, M., Furber, S. B., Davison, A. J., et al. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460558 · Cited by 79
Self-citationFunk, N., Tarrio, J., Papatheodorou, S., Popović, M., Alcantarilla, P. F., & Leutenegger, S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3061989 · Cited by 78
Real-time surface light-field capture for augmentation of planar specular surfaces
Jachnik, J., Newcombe, R. A., & Davison, A. J. (2012) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2012.6402544 · Cited by 78
“This model could be captured with existing real-time methods such as [11, 12].”
MethodSelf-citation3D Object Reconstruction from Hand-Object Interactions
Tzionas, D., & Gall, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.90 · Cited by 78
“The TSDF volume has a dimension of 350 mm for all objects with 256 voxel resolution and 6 mm maximum voxel size.”
MethodInfluentialCollaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation
Golodetz, S., Cavallari, T., Lord, N. A., Prisacariu, V. A., Murray, D. W., & Torr, P. H. S. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2868533 · Cited by 78
“[38], which demonstrated real-time voxel-based reconstruction of a desksized scene in real time using only a consumer-grade RGB-D sensor, huge progress has been made in increasing the size of scene we are able to reconstruct [39, 42, 43, 51] and in compensating for tracking drift [17, 20, 29, 54, 55].”
BackgroundRecognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor
Lysenkov, I., Eruhimov, V., & Bradski, G. R. (2012) · Robotics: Science and Systems · doi:10.15607/rss.2012.viii.035 · Cited by 78
“[19]) to compute a clean point cloud of the object.”
BackgroundRegistration of spatio-temporal point clouds of plants for phenotyping
Chebrolu, N., Magistri, F., Läbe, T., & Stachniss, C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247243 · Cited by 77
“Registering point clouds is a common problem in a lot of disciplines and multiple techniques have been proposed for laser-based or RGB-D-based mapping systems [8, 9, 60].”
BackgroundStoiber, M., Sundermeyer, M., & Triebel, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00673 · Cited by 77
Retargeting Technical Documentation to Augmented Reality
Mohr, P., Kerbl, B., Donoser, M., Schmalstieg, D., & Kalkofen, D. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702490 · Cited by 77
“The registration of the 3D model to its physical counterpart uses an RGB-D sensor and PCL-Kinfu, which is an implementation of KinectFusion [19] using the Point Cloud Library (PCL) [23].”
MethodHierarchical Voxel Block Hashing for Efficient Integration of Depth Images
Kähler, O., Prisacariu, V. A., Valentin, J., & Murray, D. W. (2015) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2015.2512958 · Cited by 77
“Each incoming image I D from the depth sensor is first aligned using a camera tracker, as in KinectFusion [1] or similar systems.”
BackgroundMethodResultInfluentialDOVE: Learning Deformable 3D Objects by Watching Videos
Wu, S., Jakab, T., Rupprecht, C., & Vedaldi, A. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01819-5 · Cited by 76
“Classic SfM methods [55, 56] use multiple views of a rigid scene, with pipelines such as KinectFusion [57] and DynamicFusion [58] integrating depth sensors for reconstructing dense static and deformable surfaces.”
MethodScanning and tracking dynamic objects with commodity depth cameras
Dou, M., Fuchs, H., & Frahm, J. (2013) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2013.6671769 · Cited by 76
“We use a similar data fusion pipeline as KinectFusion [21] for a static scene.”
BackgroundMethodInfluentialA critique of current developments in simultaneous localization and mapping
Huang, S., & Dissanayake, G. (2016) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881416669482 · Cited by 70
“Recently, truncated signed distance function (TSDF) representation has become a popular method for building dense maps, especially when RGB-D sensors are used.(88) Although promising, the uncertainty involved in the map representation cannot be handled elegantly, especially when closing large loops.”
MethodSurface-Based Exploration for Autonomous 3D Modeling
Song, S., & Jo, S. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460862 · Cited by 77
“The TSDFs can filter sensor noise from a large amount of sensing data and quickly estimate the surface point cloud with the zerocrossing method [7].”
BackgroundMethodInfluentialDeepContext: Context-Encoding Neural Pathways for 3D Holistic Scene Understanding
Zhang, Y., Bai, M., Kohli, P., Izadi, S., & Xiao, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.135 · Cited by 69
“2We use a 128× 128× 64 grid for the TSDF of a whole scene, with a voxel unit size of 0.05 meters and the truncation set at 0.15 meters. office area, on the other hand, usually includes a computer on a desk with a nearby chair.”
MethodInfluentialSelf-citationHand3D: Hand Pose Estimation using 3D Neural NetworkPreprint
Deng, X., Yang, S., Zhang, Y., Tan, P., Chang, L., & Wang, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02224 · Cited by 76
“We convert the input depth to a truncated signed distance function (TSDF [10]) representation built on this coordinate and feed it into the TSDF refine-ment network, which removes the artifacts caused by noisy and missing depth.”
MethodStructure Aware SLAM Using Quadrics and Planes
Hosseinzadeh, M., Latif, Y., Pham, T. T., Sünderhauf, N., & Reid, I. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_26 · Cited by 75
“SLAM with cameras has also seen advancement in theory and good implementations that have lead to many real-time systems from sparse ([14],[2]) to semi-dense ([3], [15]) to fully dense ([4], [6], [5]).”
BackgroundMethod
Metadata
- Authors
- Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S.
- Affiliations
- Imperial College London (GB)
- Microsoft (United States) (US)
- Venue
- 2011 10th IEEE International Symposium on Mixed and Augmented Reality
- Publisher
- IEEE
- Published
- 2011-10
- Pages
- 127-136
- ISSN
- 1554-7868 (ISSN-L)
- OpenAlex
- W1987648924
- Cited by
- 4,008
- References
- 34
- Citations by year
- 2026: 120 · 2025: 200 · 2024: 213 · 2023: 252 · 2022: 231 · 2021: 299 · 2020: 314 · 2019: 357 · 2018: 383 · 2017: 361 · 2016: 322 · 2015: 343 · 2014: 256 · 2013: 243 · 2012: 112
- FWCI
- 378.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR (publishedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- KinectFusion, real-time 3D reconstruction, depth camera, Kinect sensor, iterative closest point, pose tracking, indoor scene reconstruction, augmented reality, GPU rendering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.