KinectFusion: Real-time dense surface mapping and tracking
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103 · cited 2×
“The most important class of algorithms have been based on the ICP concept introduced in [3] which poses data alignment as a nonlinear optimisation problem in which correspondences between scans are approximated using the closest pairs of points found between scans at the previous iteration.”
BackgroundMethodMarching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104 · cited 1×
“One option is to extract the connected surfaces using a marching cubes type algorithm [18], followed by a standard rasterising rendering pipeline.”
MethodBilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“We apply a bilateral filter [27] to the raw depth map to obtain a discontinuity preserved depth map with reduced noise Dk,”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 3×
“By using only depth data, our system can work in complete darkness mitigating any issues concerning low light conditions, problematic for passive camera [17, 19, 26] and RGB-D based systems [14].”
BackgroundMethodObject modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 6×
“A related non-parametric representation used in graphics is the signed distance function (SDF) introduced in [7] for the purpose of fusing partial depth scans while mitigating problems related to mesh-based reconstruction algorithms.”
BackgroundMethodInfluentialA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“In computer vision, research on structure from motion (SFM) and multi-view stereo (MVS) has produced many compelling re-sults, in particular accurate camera tracking and sparse reconstructions (e.g. [10]), and increasingly reconstruction of dense surfaces (e.g. [24]).”
BackgroundIterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,402 · cited 1×
“A number of ICP variants perform early iterations on a subset of possibly corresponding points or operate within a coarse-to-fine scheme [31], speeding up both the data association and final pose optimisation.”
BackgroundKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 1×
“In [16] we discuss all these possibilities in detail.”
BackgroundSelf-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
Self-citation
Research works printed but not linked: 4
- weise, 0, Inhand scanning with online loop closure, Proceedings of the IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM) 2009
- wurm, 0, OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems, Proceedings of the ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation 2010
- 0, NVIDIA OptiX 2 Ray Tracing Engine
- zach, 0, A globally optimal algorithm for robust TV-L1 range image integration, Proceedings of the International Conference on Computer Vision (ICCV) 2007
Cited by
Items 801–900 of 1,006
SDF-2-SDF: Highly Accurate 3D Object Reconstruction
Slavcheva, M., Kehl, W., Navab, N., & Ilić, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_41 · Cited by 51
“– ICP-based approaches: PCL’s [27] frame-to-frame generalized ICP [33] (GICP) and frame-to-model KinectFusion [24] (KinFu); – point-to-implicit methods: frame-to-model [5] (FM-pt-SDF, available as a ROS package [4]) and our frame-to-frame modification (FF-pt-SDF ); – visual odometry: DVO [15] (available online [14]) o…”
BackgroundMethodInfluentialAugmented Vehicular Reality: Enabling Extended Vision for Future Vehicles
Qiu, H., Ahmad, F., Govindan, R., Gruteser, M., Bai, F., & Kar, G. (2017) · IEEE Workshop on Mobile Computing Systems and Applications (WMCSA) · doi:10.1145/3032970.3032976 · Cited by 51
“Kinect [20] can also produce high-quality 3D scan of an indoor environment using infrared.”
BackgroundZhao, C., Sun, L., & Stolkin, R. (2017) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2017.8023499 · Cited by 51
Parallax360: Stereoscopic 360° Scene Representation for Head-Motion Parallax
Luo, B., Xu, F., Richardt, C., & Yong, J. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2794071 · Cited by 51
Object-aware guidance for autonomous scene reconstruction
Liu, L., Xia, X., Sun, H., Shen, Q., Xu, J., Chen, B., Huang, H., & Xu, K. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201295 · Cited by 51
3-D Reconstruction of Human Body Shape From a Single Commodity Depth Camera
Zhao, T., Li, S., Ngan, K. N., & Wu, F. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2844087 · Cited by 51
“KinectFusion [22] is a commonly-used algorithm to reconstruct 3-D shape using depth cameras.”
MethodRobust Robot Pose Estimation for Challenging Scenes With an RGB-D Camera
Yu, H., Fu, Q., Yang, Z., Tan, L., Sun, W., & Sun, M. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2884321 · Cited by 51
“[31] proposed OCV ICP, which was inspired by the point cloud registration algorithm of KinectFusion [32].”
MethodReal-Time Dense Mapping for Self-Driving Vehicles using Fisheye Cameras
Cui, Z., Heng, L., Yeo, Y. C., Geiger, A., Pollefeys, M., & Sattler, T. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793884 · Cited by 51
“[16] convert a depth map into a compact stixel representation; this representation only works well for urban environments with strong planarity features unlike a TSDF volume representation [17, 25] which can represent arbitrary environments.”
BackgroundHu, P., Ho, E. S. L., & Munteanu, A. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3076340 · Cited by 51
“The pioneering work in this direction is given by KinectFusion [8], which proposes a realtime 3D reconstruction algorithm for complex and arbitrary indoor scenes.”
Mixed reality applications in urology: Requirements and future potential
Reis, G., Yılmaz, M., Rambach, J. R., Pagani, A., Suarez-Ibarrola, R., Miernik, A. J., Lesur, P., & Minaskan, N. (2021) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2021.102394 · Cited by 51
“Some of these issues can be mitigated with the use of binocular camera systems or additional inertial or depth sensors [2,8].”
BackgroundTang, J., Han, X., Tan, M., Tong, X., & Jia, K. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3087358 · Cited by 51
“solved by local correspondence matching [3], [4], [5], followed by a global optimization [6], [7], [8], it has been recently shown that, by leveraging the great modeling capacities of deep networks, the 3D surface shapes of generic objects can be learned and reconstructed from as few as a single image [9], [10], [11],…”
BackgroundContinual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observations
Yan, Z., Tian, Y., Shi, X., Guo, P. Y., Wang, P., & Zha, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01549 · Cited by 52
“RoutedFusion [64] follows the conventional TSDF fusion pipeline and learns how volumetric TSDF is updated.”
MethodPQ-Transformer: Jointly Parsing 3D Objects and Layouts From Point Clouds
Chen, X., Zhao, H., Zhou, G., & Zhang, Y. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3143224 · Cited by 51
Video-Based Outdoor Human Reconstruction
Zhu, H., Liu, Y., Fan, J., Dai, Q., & Cao, X. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2596118 · Cited by 51
“We use a Kinect to scan the full body and reconstruct the mesh model using algorithm [44], which is the modified approach based on Kinect Fusion [28].”
BackgroundMethodGlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAMPreprint
Zhang, G., Sandström, E., Zhang, Y. M., Patel, M., Gool, L. V., & Oswald, M. R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.19549 · Cited by 51
“…optimizing a neural implicit encoding of the scene in the form of either weights of an MLP [1,42,50,62] or as features anchored in dense grids [3,30, 47,56,63,73,74,89,90], hierarchical octrees [79], via voxel hashing [8,43,54,84,85], point clouds [18,33,55] or axis aligned feature planes [36,51].”
MethodWang, J., Gong, Z., Tao, B., & Yin, Z. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3156205 · Cited by 50
Submap-Based Bundle Adjustment for 3D Reconstruction from RGB-D Data
Maier, R. L., Sturm, J., & Cremers, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11752-2_5 · Cited by 50
“Real-time 3D reconstruction from images has a long history in computer vision [2,5,7,10,13,17].”
BackgroundMethodCreating automatically aligned consensus realities for AR videoconferencing
Lehment, N. H., Merget, D., & Rigoll, G. (2014) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2014.6948428 · Cited by 50
Submap matching for stereo-vision based indoor/outdoor SLAM
Brand, C., Schuster, M. J., Hirschmüller, H., & Suppa, M. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7354182 · Cited by 50
“fusion [17] is a popular algorithm for the generation of consistent 3D RGBD maps and smooth 3D reconstruction, using a frame-to-frame ICP registration.”
BackgroundMethodOmnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps
Pintore, G., Garro, V., Ganovelli, F., Gobbetti, E., & Agus, M. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477631 · Cited by 50
“In this paper, we introduce an extremely light-weight method to quickly capture and recover 2.5D multi-room indoor environments scaled to real-world metric dimensions (see Fig.”
Method3D Mapping for Visualization of Rigid Structures: A Review and Comparative Study
Santoso, F., Garratt, M., Pickering, M. R., & Asikuzzaman, M. (2015) · IEEE Sensors Journal · doi:10.1109/jsen.2015.2498940 · Cited by 50
3D Tracking of Human Hands in Interaction with Unknown Objects
Panteleris, P., Kyriazis, N., & Argyros, A. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.123 · Cited by 50
“3) to exclude these voxels of the TSDF volume from being updated with invalid information.”
BackgroundMethodInfluential3D attention-driven depth acquisition for object identification
Xu, K., Shi, Y., Zheng, L., Zhang, J., Liu, M., Huang, H. F., Su, H., Cohen‐Or, D., & Chen, B. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980224 · Cited by 50
“With the fast development of commodity depth cameras and realtime reconstruction techniques (e.g., [Newcombe et al. 2011; Nießner et al. 2013]), online scene analysis during the scanning process becomes increasingly popular, due to its advantages facilitating instant user interaction [Zhang et al.…”
BackgroundMethodSucar, E., & Hayet, J.-B. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461178 · Cited by 50
RESLAM: A real-time robust edge-based SLAM system
Schenk, F., & Fraundorfer, F. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794462 · Cited by 50
“Kintinuous [16] addresses several limitations of [14], while the surfel-based ElasticFusion [6] achieves global consistency through non-rigid deformations of the map instead of a pose graph optimization.”
BackgroundMethodRobust Non-Rigid Registration with Reweighted Position and Transformation Sparsity
Li, K., Yang, J., Lai, Y., & Guo, D. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2832136 · Cited by 50
An application independent review of multimodal 3D registration methods
Saiti, E., & Theoharis, T. (2020) · Computers & Graphics · doi:10.1016/j.cag.2020.07.012 · Cited by 50
“The dataset provides a dense 3D model per scene in TSDF format [87] and the ’ground truth’ was obtained by an implementation of the KinectFusion system [88,89] .”
MethodUncertainty-Aware Deep Multi-View Photometric Stereo
Kaya, B., Kumar, S., de Oliveira, C. E. P., Ferrari, V., & Gool, L. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01227 · Cited by 50
Stereo Matching—State-of-the-Art and Research Challenges
Bleyer, M., & Breiteneder, C. J. (2013) · Advances in computer vision and pattern recognition · doi:10.1007/978-1-4471-5520-1_6 · Cited by 50
Kamarudin, K., Zakaria, S. M. M. S., Shakaff, A. M., & Zakaria, A. (2014) · Sensors · doi:10.3390/s141223365 · Cited by 50
“Most researchers take advantage of the RGB camera and the 3D depth sensor available on the Kinect to perform 3D or RGB-Depth (RGB-D) mapping [12–17].”
MethodReal-time 3D reconstruction techniques applied in dynamic scenes: A systematic literature review
Ingale, A. K., & Udayan, J. D. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100338 · Cited by 49
Fast MRF Optimization with Application to Depth Reconstruction
Chen, Q., & Koltun, V. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.500 · Cited by 49
“It is common in practice to rectify a range image by applying a bilateral filter [20, 21, 7].”
BackgroundMethodA Novel Stereo Vision Measurement System Using Both Line Scan Camera and Frame Camera
Liu, Z., Wu, S., Wu, Q., Quan, C., & Ren, Y. (2018) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2018.2880080 · Cited by 49
Dynamic application reconfiguration on heterogeneous hardware
Fumero, J. J., Papadimitriou, M., Zakkak, F. S., Xekalaki, M., Clarkson, J., & Kotselidis, C. (2019) · ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE) · doi:10.1145/3313808.3313819 · Cited by 49
“Namely, the six benchmarks we use are Saxpy , Montercarlo , RenderTrack , BlackScholes , NBody , and DFT (Discrete Fourier Transform), and the computer vision application is the Kinect Fusion (KFu-sion) [31] implementation provided by SLAMBench [30].”
MethodActive Scene Understanding via Online Semantic Reconstruction
Zheng, L., Zhu, C., Zhang, J., Zhao, H., Huang, H. F., Nießner, M., & Xu, K. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13820 · Cited by 49
From Marker to Markerless in Augmented Reality
Oufqir, Z., Abderrahmani, A. E., & Satori, K. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-0947-6_57 · Cited by 49
Herrera-Granda, E. P., Torres, J. C., & Peluffo-Ordóñez, D. H. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37356 · Cited by 49
Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark
Chen, Z., Gebru, I. D., Richardt, C., Kumar, A., Laney, W., Owens, A., & Richard, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02067 · Cited by 48
“Many approaches that focus on 3D scene reconstruction use representations such as (truncated) signed distance fields to combine multiple observations from RGB-D sensors [43, 44, 58, 69] or standard color videos [20, 25, 41, 59].”
BackgroundObject Tracking With a Range Camera for Augmented Reality Assembly Assistance
Radkowski, R. (2015) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4031981 · Cited by 48
From one to crowd: a survey on crowdsourcing-based wireless indoor localization
Zhou, X., Чен, Т., Guo, D., Teng, X., & Yuan, B. (2017) · Frontiers of Computer Science · doi:10.1007/s11704-017-6520-z · Cited by 48
“KinectFusion [89] uses a surface reconstruction-based method on a voxel grid containing the truncated signed distance [90] to the surface.”
MethodYang, J., Xiao, Y., & Cao, Z. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2813083 · Cited by 48
“, the KinectFusion system [19], require the timing order of the input sequences.”
MethodEfficient Velodyne SLAM with point and plane features
Grant, W. S., Voorhies, R. C., & Itti, L. (2018) · Autonomous Robots · doi:10.1007/s10514-018-9794-6 · Cited by 48
“For example, the KinectFusion algorithm of Newcombe et al. (2011) can create dense reconstructions of small indoor environments using a Microsoft Kinect (or equivalent RGB-D sensor) and an array of high powered graphics processing units (GPU).”
MethodA Lightweight Approach for On-the-Fly Reflectance Estimation
Kim, K., Gu, J., Tyree, S., Molchanov, P. A., NieBner, M., & Kautz, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.12 · Cited by 48
“While 3D geometry recovery has achieved high accuracy, especially with recent RGBD-based scanning-andfusion approaches [7, 22, 33], surface reflectance estima(a) (b) (c)”
BackgroundTask-Driven Perception and Manipulation for Constrained Placement of Unknown Objects
Mitash, C., Shome, R., Wen, B., Boularias, A., & Bekris, K. E. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3006816 · Cited by 48
Progressive Point Cloud Upsampling via Differentiable Rendering
Zhang, P., Wang, X., Ma, L., Wang, S., Kwong, S., & Jiang, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3100134 · Cited by 48
“The earlier works focused on reconstructing a piece-wise smooth representation of the original shape, and specialized priors have been incorporated to address the challenges from data imperfections [19], [20], [21].”
BackgroundNeural Free-Viewpoint Performance Rendering under Complex Human-object Interactions
Sun, G., Chen, X., Chen, Y., Pang, A., Lin, P., Jiang, Y., Xu, L., Yu, J., & Wang, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475442 · Cited by 48
“human, people recover them from depthmaps [39], implicit fields [35], or semantic parts [8].”
BackgroundEfficient and robust radiance transfer for probeless photorealistic augmented reality
Gruber, L., Langlotz, T., Sen, P., Hoherer, T., & Schmalstieg, D. (2014) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2014.6802044 · Cited by 48
“In particular, this work introduces a novel approach for probeless photometric registration by relying only on input from a common RGBD camera to compute both dynamic geometry [21] and the environment light considering occlusions.”
MethodInfluentialA Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation
Nguyen, D. T., Hua, B.-S., Yu, L.-F., & Yeung, S.-K. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2772238 · Cited by 42
Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty
Proença, P. F., & Gao, Y. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.02.018 · Cited by 48
ThermalGaussian: Thermal 3D Gaussian SplattingPreprint
Lu, R., Chen, H., Zhu, Z., Qin, Y., Lu, M. X., Zhang, L., Yan, C. C., & Xue, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.07200 · Cited by 48
“In the first stage, RGB images and traditional multi-view geometry methods (Newcombe et al., 2011) are used to achieve a 3D geometric reconstruction.”
PointShopAR: Supporting Environmental Design Prototyping Using Point Cloud in Augmented Reality
Wang, Z., Nguyen, C., Asente, P., & Dorsey, J. O. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580776 · Cited by 47
DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume
Miao, X., Bai, Y., Duan, H., Huang, Y., Wan, F., Xu, X., Long, Y., & Zheng, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3305776 · Cited by 47
“Currently, depth information plays a significant role in several fields, including autonomous vehicles [1], robots [2], AR/VR applications [3], and 3D reconstructions [4].”
BackgroundSurfelNeRF: Neural Surfel Radiance Fields for Online Photorealistic Reconstruction of Indoor Scenes
Gao, Y., Cao, Y., & Shan, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00019 · Cited by 47
“Classical visual simultaneous localization and mapping (SLAM) systems [6, 12, 16, 24, 41, 43] can perform real-time 3D scene reconstruction.”
BackgroundEnd-to-End RGB-D SLAM With Multi-MLPs Dense Neural Implicit Representations
Li, M., He, J., Wang, Y., & Wang, H. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3311365 · Cited by 47
MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction
Tang, Y., Zhang, J., Yu, Z., Wang, H., & Xu, K. (2023) · ACM Transactions on Graphics · doi:10.1145/3618363 · Cited by 47
A Dense and Direct Approach to Visual Servoing Using Depth Maps
Teulière, C., & Marchand, É. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2014.2325991 · Cited by 47
“Many variants have been proposed in the literature [23], and the development of so-called RGB-D cameras has attracted a lot of attention to these methods in recent years [13], [21], [22], [27].”
BackgroundComprehensive Use of Curvature for Robust and Accurate Online Surface Reconstruction
Lefloch, D., Kluge, M., Sarbolandi, H., Weyrich, T., & Kolb, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2648803 · Cited by 47
TetraTSDF: 3D Human Reconstruction From a Single Image With a Tetrahedral Outer Shell
Onizuka, H., Hayirci, Z., Thomas, D., Sugimoto, A., Uchiyama, H., & Taniguchi, R.-I. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00605 · Cited by 47
Scalable image-based indoor scene rendering with reflections
Xu, J., Wu, X., Zhu, Z., Huang, Q., Yang, Y., Bao, H., & Xu, W. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459849 · Cited by 47
TightCap: 3D Human Shape Capture with Clothing Tightness Field
Chen, X., Pang, A., Yang, W., Wang, P., Xu, L., & Yu, J. (2021) · ACM Transactions on Graphics · doi:10.1145/3478518 · Cited by 47
“The former approaches [Collet et al. 2015; Furukawa et al. 2013; Newcombe et al. 2011b; Strecha et al. 2008] employ correspondence matching and triangulation.”
Location-Based Mixed and Augmented Reality Storytelling
Barfield, W. (2015) · CRC Press eBooks · doi:10.1201/b18703-16 · Cited by 47
Plenoptic Signal Processing for Robust Vision in Field Robotics
Dansereau, D. G. (2013) · The Sydney eScholarship Repository (The University of Sydney) · Cited by 47
Video Based Reconstruction of 3D People ModelsPreprint
Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons‐Moll, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04758 · Cited by 47
“A number of methods adapt KinectFusion for human body scanning [58, 39, 79, 17].”
BackgroundMethodInfluentialEdge-SLAM: edge-assisted visual simultaneous localization and mapping
Ben Ali, A. J., Hashemifar, Z. S., & Dantu, K. (2020) · Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3386901.3389033 · Cited by 46
Rong, V., Chen, J., Bahmani, S., Kutulakos, K. N., & Lindell, D. B. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00346 · Cited by 46
Fast Visual Odometry for 3-D Range Sensors
Jaimez, M., & González-Jiménez, J. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2428512 · Cited by 46
“A remarkable case is KinectFusion [26], which introduces a signed distance function (SDF) to represent the observed scene and applies ICP to align the depth frames against this scene model.”
Model-based furniture recognition for building semantic object maps
Günther, M., Wiemann, T., Albrecht, S., & Hertzberg, J. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2014.12.007 · Cited by 46
Retargeting Video Tutorials Showing Tools With Surface Contact to Augmented Reality
Mohr, P., Mandl, D., Tatzgern, M., Veas, E. E., Schmalstieg, D., & Kalkofen, D. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025688 · Cited by 46
“Creating a 3D reconstruction of an existing physical object is relatively straight forward using consumer depth sensors [35] or even mobile phone cameras [37].”
BackgroundA Stable and Accurate Marker-Less Augmented Reality Registration Method
Gao, Q. H., Wan, T. R., Tang, W. J., & Chen, L. F. (2017) · International Conference on Cyberworlds (CW) · doi:10.1109/cw.2017.44 · Cited by 46
“The KinectFusion framework [12] is well known for the real-time reconstruction of dense map obtained from RGBD sensors, then the ElasticFusion algorithm [13] can be used to achieve the fast and accurate real time reconstruction.”
MethodDong, W., Wang, Q., Wang, X., & Zha, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_43 · Cited by 46
Lightweight Visual Odometry for Autonomous Mobile Robots
Aladem, M., & Rawashdeh, S. A. (2018) · Sensors · doi:10.3390/s18092837 · Cited by 46
“[18] is one of earliest and most well-known RGB-D SLAM systems.”
BackgroundZhang, H., Bo, Z.-H., Yong, J., & Xu, F. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322998 · Cited by 46
“We compare our reconstruction performance with KinectFusion [Newcombe et al. 2011] on a challenging sequence, rotating a cube.”
ResultBujanca, M., Gafton, P., Sajad Saeedi, G., Nisbet, A. P., Bodin, B., OaBoyle Michael, F. P., Davison, A. J., Kelly, P. H. J., Riley, G. D., Lennox, B., et al. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794369 · Cited by 46
“DynamicFusion generalises the KinectFusion [46] pipeline to work in dynamic settings.”
BackgroundMethodInfluentialSelf-citationGround and Aerial Collaborative Mapping in Urban Environments
He, J., Zhou, Y., Huang, L., Kong, Y., & Cheng, H. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3032054 · Cited by 46
“The dense 3D map representation can be generated using surface reconstruction techniques [6], [7] with camera poses and depth point clouds as input.”
MethodRobust and Efficient RGB-D SLAM in Dynamic Environments
Yang, X. Y., Yuan, Z., Zhu, D., Chi, C., Li, K., & Liao, C. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3038323 · Cited by 46
Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo
Adam, A., Dann, C., Yair, O., Mazor, S., & Nowozin, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2567379 · Cited by 46
Bajracharya, M., Ma, J., Malchano, M., Perkins, A., Rizzi, A. A., & Matthies, L. H. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696879 · Cited by 46
“More recently, structured light depth sensors have been used to build accurate local [13] and extended [14] maps of registered point clouds.”
Acquisition of customer’s tailor measurements for 3D clothing design using virtual reality devices
Vitali, A., & Rizzi, C. (2018) · Virtual and Physical Prototyping · doi:10.1080/17452759.2018.1474082 · Cited by 45
“Among them MS Kinect sensors v1 and v2 are the most representative (Newcombe et al .”
BackgroundRadanović, M., Khoshelham, K., & Fraser, C. S. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2023.101960 · Cited by 45
Shoukat, M. U., Yan, L., Deng, D., Imtiaz, M., Safdar, M., & Nawaz, S. A. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102370 · Cited by 45
GS 3 LAM: Gaussian Semantic Splatting SLAM
Li, L., Zhang, L., Wang, Z., & Shen, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680739 · Cited by 45
“To date, existing semantic SLAM systems based on explicit scene representations often resort to points/surfels [15, 23, 26, 28], grids [16], or voxels [8, 11, 17] to construct maps.”
InfluentialVINGS-Mono: Visual-Inertial Gaussian Splatting Monocular SLAM in Large Scenes
Wu, K., Zhang, Z., Tie, M., Ai, Z., Gan, Z., & Ding, W. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3613536 · Cited by 46
“Compared with traditional explicit map representation such as voxel grids [17], surfels [18], and pointclouds [14], [19], GS representation provides a denser map and NVS capabilities.”
Hu, S., & Martin, R. R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34263-9 · Cited by 45
Instant indirect illumination for dynamic mixed reality scenes
Lensing, P., & Broll, W. (2012) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2012.6402547 · Cited by 45
“introduced with their KinectFusion project [15] an interesting application for RGB-D sensing to construct a dense representation of the real scene.”
BackgroundToward online 3-D object segmentation and mapping
Herbst, E. V., Henry, P., & Fox, D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907318 · Cited by 45
“We apply frame-to-map alignment, shown by Newcombe et al. [1] to improve single-map alignment, to alignment between multiple maps.”
BackgroundMethodInfluentialA Contour Completion Model for Augmenting Surface Reconstructions
Silberman, N., Shapira, L., Gal, R., & Kohli, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_32 · Cited by 45
“We use the KinectFusion [16] method to compute this reconstruction which also assigns a surface normal and truncated signed distance function (TSDF) [2] to the voxels.”
BackgroundMethodInfluentialRobust edge-based visual odometry using machine-learned edges
Schenk, F., & Fraundorfer, F. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202305 · Cited by 45
“KinectFusion [18] directly uses the depth images from an RGBD sensor to estimate frame-to-model motion and was later extended to a full SLAM system by Whelan et al.”
MethodMulti-Robot Active Mapping via Neural Bipartite Graph Matching
Ye, K., Dong, S., Fan, Q., Wang, H., Yi, L., Xia, F., Wang, J., & Chen, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01442 · Cited by 45
“With the fast development of range sensors (Kinect, RealSense), many scene mapping approaches [23, 29, 11, 15] are developed to em-”
BackgroundComparison of Kinect and Terrestrial LiDAR Capturing Natural Karst Cave 3-D Objects
Hämmerle, M., Höfle, B., Fuchs, J., Schröder‐Ritzrau, A., Vollweiler, N., & Frank, N. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2313599 · Cited by 45
“A permanent comparison and adjustment between predicted and current sensor measurement is performed online in real time to minimize the difference between two point clouds generated from consecutive single frames [3], [12].”
BackgroundTian, G., Liu, L., Ri, J., Liu, Y., & Sun, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.01.088 · Cited by 45
On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks
Jung, H., Ruhkamp, P., Zhai, G., Brasch, N., Li, Y., Verdié, Y., Song, J., Zhou, Y., Armagan, A., Ilić, S., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00082 · Cited by 44
“For comparison, a Kinect Azure camera induces a standard deviation of 17 mm in its working range [35].”
BackgroundMethodResultInfluentialLow-cost commodity depth sensor comparison and accuracy analysis
Breuer, T., Bodensteiner, C., & Arens, M. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2067155 · Cited by 44
Dense 3D reconstruction combining depth and RGB information
Pan, H., Guan, T., Luo, Y., Duan, L., Tian, Y., Liu, Y., Zhao, Y., & Yu, J. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.10.104 · Cited by 44
“In the future research, we will try to extend the effective distance beyond 4 meters and integrate more recent studies, including [15], [25], [28], [30], and [38], to further improve the performance of this system.”
BackgroundA novel system for object pose estimation using fused vision and inertial data
Li, J., Besada, J. A., Bernardos, A. M., Tarrío, P., & Casar, J. R. (2016) · Information Fusion · doi:10.1016/j.inffus.2016.04.006 · Cited by 44
“Sim ilarly, authors in [22] have designed a system composed by an infrared dot-pattern projector and an infrared camera.”
BackgroundFast and Effective Loop Closure Detection to Improve SLAM Performance
Güçlü, O., & Can, A. B. (2017) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-017-0718-z · Cited by 44
“Thus, most of them (such as [16, 26, 35–37]) are implemented on GPU to perform the pixel-wise operations in parallel.”
BackgroundMethodKrig, S. (2014) · Apress eBooks · doi:10.1007/978-1-4302-5930-5_2 · Cited by 44
Connecting the Dots: Learning Representations for Active Monocular Depth Estimation
Riegler, G., Liao, Y., Donné, S., Koltun, V., & Geiger, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00781 · Cited by 44
“With the introduction of the Microsoft Kinect, active consumer depth cameras have greatly impacted the field of computer vision, leading to algorithmic innovations [13, 28] and novel 3D datasets [6, 7, 35, 37], especially in the context of indoor environments.”
BackgroundMethodInterest Point Detector and Feature Descriptor Survey
Krig, S. (2016) · Springer eBooks · doi:10.1007/978-3-319-33762-3_6 · Cited by 44
Wearable Augmented Reality: Research Trends and Future Directions from Three Major Venues
Tran, T. T. M., Brown, S., Weidlich, O., Billinghurst, M., & Parker, C. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3320231 · Cited by 44
“The approaches and techniques for dense 3D reconstruction have evolved over time, building upon seminal work such as that of Newcombe et al. [77], which used depth maps from a low-cost RGB-D camera to achieve accurate real-time mapping of complex indoor scenes.”
MethodInteractive 3D Model Acquisition and Tracking of Building Block Structures
Miller, A. E., White, B., Charbonneau, E., Kanzler, Z., & LaViola, J. J. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.48 · Cited by 44
“KinectFusion [15, 26], on the 656 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL.”
Method
Metadata
- Authors
- Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S.
- Affiliations
- Imperial College London (GB)
- Microsoft (United States) (US)
- Venue
- 2011 10th IEEE International Symposium on Mixed and Augmented Reality
- Publisher
- IEEE
- Published
- 2011-10
- Pages
- 127-136
- ISSN
- 1554-7868 (ISSN-L)
- OpenAlex
- W1987648924
- Cited by
- 4,008
- References
- 34
- Citations by year
- 2026: 120 · 2025: 200 · 2024: 213 · 2023: 252 · 2022: 231 · 2021: 299 · 2020: 314 · 2019: 357 · 2018: 383 · 2017: 361 · 2016: 322 · 2015: 343 · 2014: 256 · 2013: 243 · 2012: 112
- FWCI
- 378.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR (publishedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- KinectFusion, real-time 3D reconstruction, depth camera, Kinect sensor, iterative closest point, pose tracking, indoor scene reconstruction, augmented reality, GPU rendering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.