KinectFusion: Real-time dense surface mapping and tracking
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103 · cited 2×
“The most important class of algorithms have been based on the ICP concept introduced in [3] which poses data alignment as a nonlinear optimisation problem in which correspondences between scans are approximated using the closest pairs of points found between scans at the previous iteration.”
BackgroundMethodMarching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104 · cited 1×
“One option is to extract the connected surfaces using a marching cubes type algorithm [18], followed by a standard rasterising rendering pipeline.”
MethodBilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“We apply a bilateral filter [27] to the raw depth map to obtain a discontinuity preserved depth map with reduced noise Dk,”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 3×
“By using only depth data, our system can work in complete darkness mitigating any issues concerning low light conditions, problematic for passive camera [17, 19, 26] and RGB-D based systems [14].”
BackgroundMethodObject modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 6×
“A related non-parametric representation used in graphics is the signed distance function (SDF) introduced in [7] for the purpose of fusing partial depth scans while mitigating problems related to mesh-based reconstruction algorithms.”
BackgroundMethodInfluentialA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“In computer vision, research on structure from motion (SFM) and multi-view stereo (MVS) has produced many compelling re-sults, in particular accurate camera tracking and sparse reconstructions (e.g. [10]), and increasingly reconstruction of dense surfaces (e.g. [24]).”
BackgroundIterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,402 · cited 1×
“A number of ICP variants perform early iterations on a subset of possibly corresponding points or operate within a coarse-to-fine scheme [31], speeding up both the data association and final pose optimisation.”
BackgroundKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 1×
“In [16] we discuss all these possibilities in detail.”
BackgroundSelf-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
Self-citation
Research works printed but not linked: 4
- weise, 0, Inhand scanning with online loop closure, Proceedings of the IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM) 2009
- wurm, 0, OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems, Proceedings of the ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation 2010
- 0, NVIDIA OptiX 2 Ray Tracing Engine
- zach, 0, A globally optimal algorithm for robust TV-L1 range image integration, Proceedings of the International Conference on Computer Vision (ICCV) 2007
Cited by
Items 401–500 of 1,006
Online Structure Analysis for Real-Time Indoor Scene Reconstruction
Zhang, Y., Xu, W., Tong, Y., & Zhou, K. (2015) · ACM Transactions on Graphics · doi:10.1145/2768821 · Cited by 116
“In particular, the KinectFusion system [Izadi et al. 2011; Newcombe et al. 2011] has achieved great success in generating visually compelling geometries at interactive frame rates during 3D scanning.”
MethodObject Tracking by Reconstruction With View-Specific Discriminative Correlation Filters
Kart, U., Lukežič, A., Kristan, M., Kämäräinen, J., & Matas, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00143 · Cited by 116
“[32] proposed an iterative closest point (ICP) based algorithm for depth sequences for dense mapping of indoor scenes.”
BackgroundA Review of Point Set Registration: From Pairwise Registration to Groupwise Registration
Zhu, H., Guo, B., Zou, K., Li, Y., Yuen, K., Mihaylova, L. S., & Leung, H. (2019) · Sensors · doi:10.3390/s19051191 · Cited by 116
“Traditionally, this problem is performed using pairwise registration repeatedly [2], such as sequentially strategy [94–98] and one-versus-all strategy [99–101].”
MethodEdge-SLAM: Edge-Assisted Visual Simultaneous Localization and Mapping
Ali, A. J. B., Kouroshli, M., Semenova, S., Hashemifar, Z. S., Ko, S. Y., & Dantu, K. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3561972 · Cited by 117
“They build on initial work from systems such as Kinect Fusion [51], and Kintinuous [64] that first used RGB-D sensors for 3-”
MethodLRRU: Long-short Range Recurrent Updating Networks for Depth Completion
Wang, Y., Li, B., Zhang, G., Liu, Q., Gao, T., & Dai, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00864 · Cited by 116
“Acquiring accurate and dense scene depth plays a fundamental role in various applications, such as autonomous driving [33] and augmented reality [25].”
BackgroundRGBD-fusion: Real-time high precision depth recovery
Or-El, R., Rosman, G., Wetzler, A., Kimmel, R., & Bruckstein, A. M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299179 · Cited by 115
“Probably the most popular effort in this area is the KinectFusion algorithm [32], in which a real time depth stream is fused on the GPU into a truncated signed distance function to accurately describe a 3D model of a scanned volume.”
BackgroundReal-time pixel luminance optimization for dynamic multi-projection mapping
Siegl, C., Colaianni, M., Thies, L., Thies, J., Zollhöfer, M., Izadi, S., Stamminger, M., & Bauer, F. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818111 · Cited by 115
Self-citationSemanticPaint: Interactive 3D Labeling and Learning at your Fingertips
Valentin, J., Vineet, V., Cheng, M., Kim, D., Shotton, J., Kohli, P., Nießner, M., Criminisi, A., Izadi, S., & Torr, P. H. S. (2015) · ACM Transactions on Graphics · doi:10.1145/2751556 · Cited by 116
“Methods for real-time dense reconstructions, even over large physical scales, with only a single commodity depth or RGB camera have been demonstrated [Rusinkiewicz et al. 2002; Newcombe et al. 2011; Izadi et al. 2011; Newcombe et al. 2011; Chen et al. 2013; Nießner et al. 2013; Pradeep et al. 2013].”
BackgroundMethodSelf-citationEmptying, refurnishing, and relighting indoor spaces
Zhang, E., Cohen, M. F., & Curless, B. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982432 · Cited by 115
Elastic Fragments for Dense Scene Reconstruction
Zhou, Q.-Y., Miller, S., & Koltun, V. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.65 · Cited by 114
“[16], building on work on range image integration [2], real-time range scanning [23], and monocular SLAM [3, 4, 12] showed that registering each input image to a growing volumetric model can average out high-frequency error and produce smooth reconstructions of objects and small scenes.”
BackgroundMethodInfluential3D Modeling on the Go: Interactive 3D Reconstruction of Large-Scale Scenes on Mobile Devices
Schöps, T., Sattler, T., Häne, C., & Pollefeys, M. (2015) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2015.40 · Cited by 114
“, RGBD (color and depth) data, real-time volumetric reconstruction from depth maps has become feasible and popular [2, 20, 22, 30, 33].”
BackgroundMR360: Mixed Reality Rendering for 360° Panoramic Videos
Rhee, T. J., Petikam, L., Allen, B., & Chalmers, A. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2657178 · Cited by 114
“Using the RGB-D sensing camera, 3D real-world geometry can be reconstructed online [31], and the results have been adapted in MR rendering recently [13, 14, 26].”
MethodUltra-wideband aided fast localization and mapping system
Wang, C., Zhang, H., Nguyen, T.-M., & Xie, L. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205968 · Cited by 114
“Other existing methods capable of producing a real-time dense map are computationally heavy [3], [4] and are not suitable for devices with limited computational resources.”
MethodAutonomous mobile robot navigation in uneven and unstructured indoor environments
Wang, C., Meng, L., She, S., Mitchell, I. M., Li, T., Tung, F., Wan, W., Meng, M. Q. ‐., & de Silva, C. W. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202145 · Cited by 114
“However, the ICP method is only confined to desk area [19], while the local feature based methods are not robust in environments where”
BackgroundMethodAitken, J. M., Evans, M. H., Worley, R., Edwards, S., Zhang, R., Dodd, T. J., Mihaylova, L. S., & Anderson, S. (2021) · IEEE Access · doi:10.1109/access.2021.3115981 · Cited by 116
“Truncated signed distance fields have also been applied to data from depth cameras [155].”
MethodHigh-Fidelity 3D Digital Human Head Creation from RGB-D Selfies
Bao, L., Lin, X., Chen, Y., Zhang, H., Wang, S., Zhe, X., Kang, D., Huang, H., Jiang, X., Wang, J., et al. (2021) · ACM Transactions on Graphics · doi:10.1145/3472954 · Cited by 115
“First, accumulated point clouds are obtained with rigid registration [Newcombe et al. 2011].”
MethodMobile augmented reality for computer-assisted percutaneous nephrolithotomy
Müller, M., Rassweiler, M.-C., Klein, J. T., Seitel, A., Gondan, M., Baumhauer, M., Teber, D., Rassweiler, J. J., Meinzer, H.-P., & Maier‐Hein, L. (2013) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-013-0828-4 · Cited by 114
“Marker-less methods use natural feature tracking [41] or cameras equipped with a range sensor, such as Time-of-Flight ( ToF ) cameras [15,42] or the Microsoft Kinect camera [43,44].”
MethodThe Effect of View Independence in a Collaborative AR System
Tait, M., & Billinghurst, M. (2015) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-015-9231-8 · Cited by 113
“The local user ’ s physical environment is captured and modeled using the PointCloudLibrary implementation of KinectFusion (Izadi et al. 2011; Newcombe et al. 2011).”
MethodOn unifying key-frame and voxel-based dense visual SLAM at large scales
Meilland, M., & Comport, A. I. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696881 · Cited by 113
“For volumetric approaches such as [14], this is achieved by integrating the new measurements (depths and intensities) into the TSDF through a weighted running average.”
MethodHuang, X., Zhang, J., Fan, L., Wu, Q., & Yuan, C. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2695888 · Cited by 113
“The most popular direct method is the ICP [7] algorithm, which alternates between estimating the point correspondence and estimating the transformation matrix for a given correspondence [1], [11], [12].”
MethodA Generative Model for the Joint Registration of Multiple Point Sets
Evangelidis, G. D., Kounades-Bastian, D., Horaud, R., & Psarakis, E. Ζ. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_8 · Cited by 113
“Multiple point-set registration is often solved using a sequential pairwise registration strategy [6,20,17,24,8].”
MethodProximal Deep Structured Models
Wang, S., Fidler, S., & Urtasun, R. (2016) · neural information processing systems · Cited by 111
Evaluating Egomotion and Structure-from-Motion Approaches Using the TUM RGB-D Benchmark
Sturm, J., Burgard, W., & Cremers, D. (2012) · Cited by 112
Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade
Cavallari, T., Golodetz, S., Lord, N. A., Valentin, J., Prisacariu, V. A., Stefano, L. D., & Torr, P. H. S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2915068 · Cited by 112
“CAMERA pose estimation is a key computer vision problem, with applications in simultaneous localisation and mapping (SLAM) [1], [2], [3], [4], virtual and augmented reality [5], [6], [7], [8], [9], [10] and navigation [11].”
BackgroundInfluentialManhattanSLAM: Robust Planar Tracking and Mapping Leveraging Mixture of Manhattan Frames
Yunus, R., Li, Y., & Tombari, F. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9562030 · Cited by 112
“KinectFusion [27] provides an algorithm for real-time dense mapping and tracking of surfaces on a GPU using ICP”
MethodLoopy-SLAM: Dense Neural SLAM with Loop Closures
Liso, L., Sandström, E., Yugay, V., Gool, L. V., & Oswald, M. R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01925 · Cited by 114
“Online dense 3D reconstruction of scenes with an RGBD camera has been an active area of research for years [12, 34, 35, 46, 69, 75], and remains an open problem.”
Digital Restoration of Cultural Heritage With Data-Driven Computing: A Survey
Basu, A., Paul, S., Ghosh, S., Das, S., Chanda, B., Bhagvati, C., & Snåšel, V. (2023) · IEEE Access · doi:10.1109/access.2023.3280639 · Cited by 113
“The stages performed (Figure 5) for the method are defined as follows: • This stage scans the structure’s surface area using [122].”
MethodSaarinen, J., Andreasson, H., Stoyanov, T., & Lilienthal, A. J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913499415 · Cited by 110
“Finally, recent advances (Newcombe et al., 2011) in dense mapping methods utilize a volumetric representation of space, using a 3D sample grid.”
MethodA review of visual inertial odometry from filtering and optimisation perspectives
Gui, J., Gu, D., Wang, S., & Hu, H. (2015) · Advanced Robotics · doi:10.1080/01691864.2015.1057616 · Cited by 110
“KinectFusion in [27] is the one which provides a real-time solution for dense alignment via separated mapping and tracking threads in GPU where a truncated signed distance function is employed for scene model.”
BackgroundTexture Mapping for 3D Reconstruction with RGB-D Sensor
Fu, Y., Yan, Q., Yang, L., Liao, J., & Xiao, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00488 · Cited by 109
“reduce computation complexity, unlike [30], we utilize [28] to reconstruct 3D models instead of KinectFusion [18, 22], and select texture candidate images by weighting the elements of image clarity, jitter, blur and viewport overlay.”
BackgroundMethodNodeSLAM: Neural object descriptors for multi-view shape reconstruction
Sucar, E., Wada, K., & Davison, A. (2020) · 2020 International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00105 · Cited by 108
“One of the targets of visual SLAM research has been a continuous improvement in the richness of scene representations, from sparse SLAM systems which reconstruct point clouds, [4, 5] via dense surface representations [22, 36] to semantically labelled dense maps [20, 40].”
BackgroundSelf-citationObstacle detection for self-driving cars using only monocular cameras and wheel odometry
Hane, C., Sattler, T., & Pollefeys, M. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354095 · Cited by 91
“We follow a different approach which originates from the fusion of laser point clouds [24] and has since then been used successfully in many computer vision systems [25], [26], [27].”
MethodReal-time probabilistic fusion of sparse 3D LIDAR and dense stereo
Maddern, W., & Newman, P. M. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759342 · Cited by 108
“The advent of affordable RGBD sensors [1] and 3D LIDAR scanners [2] has led to significant advances over the last decade in the fields of indoor [3], [4] and outdoor [5], [6] robotics respectively.”
BackgroundSurfelMeshing: Online Surfel-Based Mesh Reconstruction
Schöps, T., Sattler, T., & Pollefeys, M. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2947048 · Cited by 108
“Voxel-basedmethods such asKinectFusion [1] commonly use the volume for both tracking andmapping.”
BackgroundMethodInfluentialMulti-View Stereo by Temporal Nonparametric Fusion
Hou, Y., Kannala, J., & Solin, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00274 · Cited by 108
“approaches are widely used for fusion and reconstruction [22, 23].”
MethodRGBD-Inertial Trajectory Estimation and Mapping for Ground Robots
Shan, Z., Li, R., & Schwertfeger, S. (2019) · Sensors · doi:10.3390/s19102251 · Cited by 108
“Different from Kinect Fusion [14], which uses GPU accelerated ICP [71] for finding matched points between two depth images, to make the system compact and suitable for small robot integration, we simply project 3D points from each depth image, which are synchronized with keyframes with their corresponding poses estima…”
MethodRealtime Computer Vision with OpenCV
Pulli, K., Baksheev, A., Kornyakov, K., & Eruhimov, V. (2012) · Queue · doi:10.1145/2181796.2206309 · Cited by 108
Virtual and Augmented Reality on the 5G Highway
Orlosky, J., Kiyokawa, K., & Takemura, H. (2017) · Journal of Information Processing · doi:10.2197/ipsjjip.25.133 · Cited by 107
“Though many systems have realized dense 3D reconstructions through techniques like DTAM [39] or KinectFusion [40], the constant sharing and synchronization of these environments between users is still a significant challenge.”
BackgroundExploiting Visibility Information in Surface Reconstruction to Preserve Weakly Supported Surfaces
Jančošek, M., & Pajdla, T. (2014) · International Scholarly Research Notices · doi:10.1155/2014/798595 · Cited by 107
“Another volumetric depth-map fusion method [27, 28] can produce high quality result fast; however, it requires many and relatively high-quality depth maps.”
BackgroundAdvances in Inference and Representation for Simultaneous Localization and Mapping
Rosen, D. M., Doherty, K., Espinoza, A. T., & Leonard, J. J. (2021) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-072720-082553 · Cited by 106
“KinectFusion: Real-Time Dense Surface Mapping and Tracking.”
BackgroundYang, J., Barde, A., & Billinghurst, M. (2022) · Journal of the Audio Engineering Society · doi:10.17743/jaes.2022.0048 · Cited by 106
KinectAvatar: Fully Automatic Body Capture Using a Single Kinect
Cui, Y., Chang, W., Nöll, T., & Stricker, D. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-37484-5_12 · Cited by 106
“[5] apply a bilateral filter [10] to the raw Kinect depth map to obtain a discontinuity preserved depth map with reduced noise.”
BackgroundMethod3D indoor scene modeling from RGB-D data: A survey
Chen, K., Lai, Y., & Hu, S. (2015) · Computational Visual Media · doi:10.1007/s41095-015-0029-x · Cited by 106
“Copyright a n d m o r al right s for p u blications m a d e available in ORCA a r e r e t ain e d by t h e copyrig ht hold e rs.”
BackgroundPatch-based optimization for image-based texture mapping
Bi, S., Kalantari, N. K., & Ramamoorthi, R. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073610 · Cited by 106
“With the availability of consumer depth cameras to the public, ordinary users are now able to produce geometric models of objects using techniques like KinectFusion [Newcombe et al. 2011]. However, reproducing the full appearance of real-world objects also requires reconstructing high-quality texture maps. Imagebased…”
MethodInfluentialMarkerless tracking system for augmented reality in the automotive industry
Lima, J. P., Roberto, R. A., Simões, F., Almeida, M. W. S., Figueiredo, L. S., Teixeira, J. M., & Teichrieb, V. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.03.060 · Cited by 106
Stotko, P., Krumpen, S., Hullin, M. B., Weinmann, M., & Klein, R. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2899231 · Cited by 105
“Earlier of these volumetric reconstruction frameworks such as KinectFusion [Newcombe et al. 2011] rely on the use of a uniform grid so that the memory requirement linearly scales with the overall grid size and not the significantly smaller subset of surface areas.”
MethodDiminished Reality Based on Image Inpainting Considering Background Geometry
Kawai, N., Sato, T., & Yokoya, N. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2462368 · Cited by 105
When 2.5D is not enough: Simultaneous reconstruction, segmentation and recognition on dense SLAM
Tateno, K., Tombari, F., & Navab, N. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487378 · Cited by 104
“The Kinect Fusion Reconstruction stages and Depth Map Segmentation stage are implemented on GPU, while the remaining segmentation stages and all object recognition stages are implemented on CPU.”
BackgroundMethodInfluentialVolk, R., Luu, T. H., Mueller-Roemer, J. S., Sevilmis, N., & Schultmann, F. (2018) · Automation in Construction · doi:10.1016/j.autcon.2018.03.017 · Cited by 90
StyleMesh: Style Transfer for Indoor 3D Scene Reconstructions
Höllein, L., Johnson, J., & Nießner, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00610 · Cited by 104
BOP: Benchmark for 6D Object Pose EstimationPreprint
Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Buch, A. G., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08319 · Cited by 105
“The 3D object models were created using KinectFusion-like systems for 3D surface reconstruction [26,33].”
MethodGO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction
Wang, J., Bleja, T., & Agapito, L. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00055 · Cited by 103
DDN-SLAM: Real Time Dense Dynamic Neural Implicit SLAM
Li, M., Guo, Z., Deng, T., Zhou, Y., Ren, Y., & Wang, H. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3546130 · Cited by 104
Active garment recognition and target grasping point detection using deep learning
Corona, E., Alenyà, G., Gabas, A., & Torras, C. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.09.042 · Cited by 90
Semantic 3D occupancy mapping through efficient high order CRFs
Yang, S., Huang, Y., & Scherer, S. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202212 · Cited by 103
“The map can be composed of different geometric elements such as points [1], planes [2], and grid voxels [3] etc.”
BackgroundTechnical Evaluation of HoloLens for Multimedia: A First Look
Liu, Y., Dong, H., Zhang, L., & Saddik, A. E. (2018) · IEEE Multimedia · doi:10.1109/mmul.2018.2873473 · Cited by 102
“The real environment can be reconstructed as a three-dimensional (3-D) model by means of the depth camera, the environmental understanding cameras, and the KinectFusion algorithm, the last of which was originally developed for 3-D reconstruction using the Kinect depth camera.(7) Then, the virtual environment component…”
MethodYu, T., Zhao, J., Zheng, Z., Guo, K., Dai, Q., Li, H., Pons‐Moll, G., & Liu, Y. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2928296 · Cited by 103
“Classical TSDF fusion methods proposed in KinectFusion [61] and DynamicFusion [14] fuse all the depth frames using a relatively uniform integration weight to complete the surface and filter depth noise.”
MethodUsing the Kinect as a navigation sensor for mobile robotics
Oliver, A., Kang, S., Wünsche, B. C., & MacDonald, B. A. (2012) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1145/2425836.2425932 · Cited by 102
“In 2011 the KinectFusion algorithm was presented, which uses the Kinect depth data to reconstruct a 3D scene using the Kinect sensor like a handheld laser scanner [11].”
MethodFutureMapping: The Computational Structure of Spatial AI SystemsPreprint
Davison, A. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11288 · Cited by 103
“Mainstream geometrical computer vision started to take advantage of parallel processing in the form of GPUs nearly 10 years ago (e.g. [45]), and in Spatial AI this led to break-throughs in dense SLAM [42, 41].”
BackgroundMethodInfluentialSelf-citationTurning Mobile Phones into 3D Scanners
Kolev, K., Tanskanen, P., Speciale, P., & Pollefeys, M. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.504 · Cited by 101
“Interactive systems based on video cameras [6] and depth sensors [14, 7] have been demonstrated.”
BackgroundBonatti, R., Ho, C., Wang, W., Choudhury, S., & Scherer, S. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968163 · Cited by 101
“Recent advances in mapping have led to better representations that can incrementally compute the truncated signed distance field (TSDF) [26], [27], i.e. return the distance and gradient to nearest object surface for a query.”
BackgroundEM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association
Strecke, M., & Stueckler, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00596 · Cited by 101
“In ICP, the projective association is only performed once and requires several lookups per pixel until a surface is found.”
BackgroundMethodInfluentialCanny-VO: Visual Odometry With RGB-D Cameras Based on Geometric 3-D–2-D Edge Alignment
Zhou, Y. B., Li, H., & Kneip, L. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2875382 · Cited by 101
“signed distance field for dense 3-D surface reconstruction [9], [34], [35].”
BackgroundA Coarse-to-Fine Algorithm for Matching and Registration in 3D Cross-Source Point Clouds
Huang, X., Zhang, J., Wu, Q., Fan, L., & Yuan, C. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2730232 · Cited by 100
“The typical example is the iterative closest point (ICP) [6] algorithm [15]–[17].”
BackgroundVPS-SLAM: Visual Planar Semantic SLAM for Aerial Robotic Systems
Bavle, H., de la Puente, P., How, J. P., & Campoy, P. (2020) · IEEE Access · doi:10.1109/access.2020.2983121 · Cited by 100
“On the other hand, other state of the art SLAM based techniques, such as [4]–[6], focus on dense 3D mapping of the environment, hence requiring high end CPU and GPU hardware in order to achieve real-time operation, which is a clear limitation on board an aerial robot with low computational…”
Background3D Depth Cameras in Vision: Benefits and Limitations of the Hardware
Kadambi, A., Bhandari, A., & Raskar, R. (2014) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-08651-4_1 · Cited by 100
Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks
Ge, L., Liang, H., Yuan, J., & Thalmann, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2827052 · Cited by 99
“For real-time considerations, the projective TSDF [58], where the closest point is found only on the line of sight in the camera frame, should be used.”
MethodRNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy
Ma, R., Wang, R., Zhang, Y., Pizer, S. M., McGill, S., Rosenman, J. G., & Frahm, J. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102100 · Cited by 99
CoRBS: Comprehensive RGB-D benchmark for SLAM using Kinect v2
Wasenmüller, O., Meyer, M., & Stricker, D. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477636 · Cited by 98
“Our novel benchmark allows for the first time to independently evaluate the localization as well as the mapping part of RGB-D SLAM systems with real data.”
BackgroundCrowd-SLAM: Visual SLAM Towards Crowded Environments using Object Detection
Soares, J. C. V., Gattass, M., & Meggiolaro, M. A. (2021) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-021-01414-1 · Cited by 98
“KinectFusion [10], for instance, uses only depth data to create a dense volumetric model, and ICP to track the camera pose.”
MethodOn the Limits of Pseudo Ground Truth in Visual Camera Re-localisation
Brachmann, E., Humenberger, M., Rother, C., & Sattler, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00616 · Cited by 99
“3 This cost function, implemented in Open3D [90], measures the 3D alignment of the two point clouds and replicates the metric minimised by algorithms such as KinectFusion [33,52] and BundleFusion [19].”
BackgroundMethodInfluentialBrachmann, E., Wynn, J., Chen, S., Cavallari, T., Monszpart, Á., Turmukhambetov, D., & Prisacariu, V. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72992-8_24 · Cited by 98
RGB-D edge detection and edge-based registration
Choi, C., Trevor, A. J. B., & Christensen, H. I. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696558 · Cited by 98
“[28] showed a dense approach called KinectFusion.”
Garon, M., & Lalonde, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2734599 · Cited by 98
“With this reference, multiple frames are fused together and ICP is used to match the reconstructed object with its 3D model.”
MethodInfluentialAldomà, A., Tombari, F., Prankl, J., Richtsfeld, A., Stefano, L. D., & Vincze, M. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630859 · Cited by 97
“Motivated by KinectFusion [16], tracked frames are aligned with a global model using Iterative Closest Point (ICP).”
Method3Dlite: towards commodity 3D scanning for content creation
Huang, J., Dai, A., Guibas, L. J., & Nießner, M. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130824 · Cited by 97
“Volumetric fusion [Curless and Levoy 1996], widely used by many state-of-the-art online and offline reconstruction methods [Chen et al. 2013; Choi et al. 2015; Dai et al. 2017b; Izadi et al. 2011; Newcombe et al. 2011; Nießner et al. 2013] is a very efficient approach to generating an implicit surface representation o…”
BackgroundMethodIndoor Scan2BIM: Building information models of house interiors
Murali, S., Speciale, P., Oswald, M. R., & Pollefeys, M. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206513 · Cited by 97
“In this paper, we propose a system that automatically generates 3D room layouts from indoor building data, either obtained from image-based matching algorithms or mobile 3D depth sensors like Google Project Tango [1], [2], Microsoft Kinect [3] or Microsoft HoloLens [4].”
MethodThe ACRV picking benchmark: A robotic shelf picking benchmark to foster reproducible research
Leitner, J., Tow, A. W., Sunderhauf, N., Dean, J. E., Durham, J. W., Cooper, M., Eich, M., Lehnert, C., Mangels, R., McCool, C., et al. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989545 · Cited by 78
“The pipline was split into three distinct stages: a Kinect Fusion (KinFu) [20] approach whilst running a scan motion, a pointcloud-based segmentation algorithm, and a classification stage using a deep convolutional network (CNN).”
MethodPatch Volumes: Segmentation-Based Consistent Mapping with RGB-D Cameras
Henry, P., Fox, D., Bhowmik, A., & Mongia, R. (2013) · 2013 International Conference on 3D Vision · doi:10.1109/3dv.2013.59 · Cited by 85
“provided an inspired approach to smaller scale high fidelity geometric reconstruction with KinectFusion [15].”
BackgroundMethodInfluentialDomain Transfer for Semantic Segmentation of LiDAR Data using Deep Neural Networks
Langer, F., Milioto, A., Haag, A., Behley, J., & Stachniss, C. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341508 · Cited by 96
“We also use a volumetric method for fusing the point clouds which is widely used in robotics, known as the truncated signed distance function or TSDF [20].”
MethodUser-Independent Intent Recognition for Lower Limb Prostheses Using Depth Sensing
Massalin, Y., Abdrakhmanova, M., & Varol, H. A. (2017) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2017.2776157 · Cited by 96
“retrieved point clouds from a moving depth camera and developed a real-time, cost-efficient and high accuracy 3D physical scene reconstruction system [19].”
BackgroundEnd-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans
Avetisyan, A., Dai, A., & Nießner, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00264 · Cited by 96
“State-of-theart 3D reconstruction approaches can now achieve impressive capture and reconstruction of real-world environments [19, 26, 27, 38, 38, 5, 8], spurring forth many potential applications of this digitization, such as content creation, or augmented or virtual reality.”
BackgroundMethodAn Integrated Platform for Live 3D Human Reconstruction and Motion Capturing
Alexiadis, D. S., Chatzitofis, A., Zioulis, N., Zoidi, O., Louizis, G., Zarpalas, D., & Daras, P. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2576922 · Cited by 95
“7, the proposed volumetric reconstruction is qualitatively compared with a Truncated Signed Distance Function (TSDF)-based reconstruction [12], [21], at the same volume resolution (2r×2r+1×2r , r = 7).”
BackgroundMethodReal-time High-accuracy Three-Dimensional Reconstruction with Consumer RGB-D Cameras
Cao, Y., Kobbelt, L. P., & Hu, S. (2018) · ACM Transactions on Graphics · doi:10.1145/3182157 · Cited by 95
“…handheld cameras is known as simultaneous localization and mapping (SLAM); in computer vision and graphics, since the pioneering KinectFusion work (Newcombe et al. 2011), many approaches, both real time and offline, have been proposed to reconstruct scenes from depth cameras or RGB-D streams…”
BackgroundMethodInfluentialLiu, Y., Wang, C., Song, Z., & Wang, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_28 · Cited by 95
Point-Plane SLAM Using Supposed Planes for Indoor Environments
Zhang, X., Wang, W., Qi, X., Liao, Z., & Wei, R. (2019) · Sensors · doi:10.3390/s19173795 · Cited by 95
“Some other point-based SLAM methods can also obtain a sparse map [9], semi-dense map [10] or even dense map [11,12].”
MethodWei, Z., Xu, Z., Guo, J., Hou, Y., Gao, C., Cai, Z., Luo, J., & Shao, L. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127754 · Cited by 95
“AdaGrasp [18] used a 3D TSDF volume [24] to encode the robot hand.”
Zahia, S., Garcia Zapirain, M. B., Sevillano, X., González, A., Kim, P. J., & Elmaghraby, A. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101742 · Cited by 84
Simultaneous Localization and Mapping in the Epoch of Semantics: A Survey
Sualeh, M., & Kim, G.-W. (2018) · International Journal of Control Automation and Systems · doi:10.1007/s12555-018-0130-x · Cited by 94
“KinectFusion [124] 11 R D ICP - + - - + -”
Sparse Depth Enhanced Direct Thermal-Infrared SLAM Beyond the Visible Spectrum
Shin, Y.-S., & Kim, A. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2923381 · Cited by 94
ClusterSLAM: A SLAM Backend for Simultaneous Rigid Body Clustering and Motion Estimation
Huang, J., Yang, S. Y., Zhao, Z., Lai, Y., & Hu, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00597 · Cited by 94
Occlusion Handling in Augmented Reality: Past, Present and Future
Macedo, M. C. F., & Apolinário, A. L. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3117866 · Cited by 94
“For instance, the work of Luo et al. [125] and the depth-based techniques that reconstruct a 3D model of the real scene [104], [105], [122] are generally able to provide temporally coherent depth maps, which result in consistent occlusion handling during the live streaming of the AR application.”
BackgroundDirect, Dense, and Deformable: Template-Based Non-rigid 3D Reconstruction from RGB Video
Yu, R., Russell, C., Campbell, N. D. F., & Agapito, L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.111 · Cited by 94
“Systems such as KinectFusion [ 20] allow users to scan the detailed 3D shape of rigid scenes.”
BackgroundOKVIS2: Realtime Scalable Visual-Inertial SLAM with Loop ClosurePreprint
Leutenegger, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.09199 · Cited by 94
“KinectFusion [17] reconstructs a volumetric Truncated Signed Distance (TSDF) map and performs geometric alignment in tracking, by application of the Iterative Closest Point (ICP) algorithm for every pixel of the live frame.”
MethodTotal Moving Face Reconstruction
Suwajanakorn, S., Kemelmacher-Shlizerman, I., & Seitz, S. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_52 · Cited by 93
Dense Continuous-Time Tracking and Mapping with Rolling Shutter RGB-D Cameras
Kerl, C., Stückler, J., & Cremers, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.261 · Cited by 93
“Alternatively, if a consistent model of the scene is available, e.g ., as signed distance function volume [12] or a set of keyframes [10], the reference frames can be synthesized from this model.”
BackgroundReal-Time Monocular Object-Model Aware Sparse SLAM
Hosseinzadeh, M., Li, K., Latif, Y., & Reid, I. D. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793728 · Cited by 93
“Denser representations [2], [3], [4], [5], [6], remain equivalent to a collection of points in this regard.”
BackgroundTactile Mapping and Localization from High-Resolution Tactile ImprintsPreprint
Bauzá, M., Canal, O., & Rodríguez, A. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 93
Ditto: Building Digital Twins of Articulated Objects from Interaction
Jiang, Z., Hsu, C.-C., & Zhu, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00553 · Cited by 92
Crowd Map: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor-Rich Videos
Chen, S., Li, M., Ren, K., & Qiao, C. (2015) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2015.9 · Cited by 92
Metadata
- Authors
- Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S.
- Affiliations
- Imperial College London (GB)
- Microsoft (United States) (US)
- Venue
- 2011 10th IEEE International Symposium on Mixed and Augmented Reality
- Publisher
- IEEE
- Published
- 2011-10
- Pages
- 127-136
- ISSN
- 1554-7868 (ISSN-L)
- OpenAlex
- W1987648924
- Cited by
- 4,008
- References
- 34
- Citations by year
- 2026: 120 · 2025: 200 · 2024: 213 · 2023: 252 · 2022: 231 · 2021: 299 · 2020: 314 · 2019: 357 · 2018: 383 · 2017: 361 · 2016: 322 · 2015: 343 · 2014: 256 · 2013: 243 · 2012: 112
- FWCI
- 378.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR (publishedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- KinectFusion, real-time 3D reconstruction, depth camera, Kinect sensor, iterative closest point, pose tracking, indoor scene reconstruction, augmented reality, GPU rendering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.