KinectFusion: Real-time dense surface mapping and tracking
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103 · cited 2×
“The most important class of algorithms have been based on the ICP concept introduced in [3] which poses data alignment as a nonlinear optimisation problem in which correspondences between scans are approximated using the closest pairs of points found between scans at the previous iteration.”
BackgroundMethodMarching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104 · cited 1×
“One option is to extract the connected surfaces using a marching cubes type algorithm [18], followed by a standard rasterising rendering pipeline.”
MethodBilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“We apply a bilateral filter [27] to the raw depth map to obtain a discontinuity preserved depth map with reduced noise Dk,”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 3×
“By using only depth data, our system can work in complete darkness mitigating any issues concerning low light conditions, problematic for passive camera [17, 19, 26] and RGB-D based systems [14].”
BackgroundMethodObject modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 6×
“A related non-parametric representation used in graphics is the signed distance function (SDF) introduced in [7] for the purpose of fusing partial depth scans while mitigating problems related to mesh-based reconstruction algorithms.”
BackgroundMethodInfluentialA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“In computer vision, research on structure from motion (SFM) and multi-view stereo (MVS) has produced many compelling re-sults, in particular accurate camera tracking and sparse reconstructions (e.g. [10]), and increasingly reconstruction of dense surfaces (e.g. [24]).”
BackgroundIterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,402 · cited 1×
“A number of ICP variants perform early iterations on a subset of possibly corresponding points or operate within a coarse-to-fine scheme [31], speeding up both the data association and final pose optimisation.”
BackgroundKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 1×
“In [16] we discuss all these possibilities in detail.”
BackgroundSelf-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
Self-citation
Research works printed but not linked: 4
- weise, 0, Inhand scanning with online loop closure, Proceedings of the IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM) 2009
- wurm, 0, OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems, Proceedings of the ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation 2010
- 0, NVIDIA OptiX 2 Ray Tracing Engine
- zach, 0, A globally optimal algorithm for robust TV-L1 range image integration, Proceedings of the International Conference on Computer Vision (ICCV) 2007
Cited by
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2705103 · Cited by 6,898
“Note that our SLAM does not perform any fusion like KinectFusion [4] or similar, but the good definition indicates the accuracy of the keyframe poses.”
BackgroundScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
“The RGB-D scanning system must be trivial to use, the data processing must be robust and automatic, the semantic annotations must be crowdsourced, and the flow of data through the system must be handled by a tracking server.”
A benchmark for the evaluation of RGB-D SLAM systems
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385773 · Cited by 3,976
“Other algorithms focus on fusing depth maps to a coherent 3D model [5].”
BackgroundPast, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“attracting increasing attention after the seminal work [183].”
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
“This inherits from the success of fusion approaches that utilize a truncated SDF (TSDF), pioneered in [15, 37], to combine noisy depth maps into a single 3D model.”
MethodSelf-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
OctoMap: an efficient probabilistic 3D mapping framework based on octrees
Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., & Burgard, W. (2013) · Autonomous Robots · doi:10.1007/s10514-012-9321-0 · Cited by 2,866
“3D SLAM system based on a low-cost depth camera and GPU processing was proposed by Newcombe et al. (2011) to reconstruct dense surfaces in indoor scenes.”
MethodLego-loam: Lightweight and ground-optimized lidar odometry and mapping on variable terrain
Shan, T., & Englot, B. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594299 · Cited by 2,295
“In addition, several ICP variants have leveraged parallel computing for improved efficiency [5]–[8].”
BackgroundKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110
“As shown in [21], this allows us to mitigate issues of drift and reduce ICP errors, by tracking directly from the raycasted model as opposed to frame-to-frame ICP tracking.”
MethodSelf-citationOctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.701 · Cited by 1,589
“ese cases, the input to the network is of two-dimensional nature, e.g., in image classification [18], object detection [35] or semantic segmentation [13]. However, recent advances in 3D reconstruction [33] and graphics [21] allow capturing and modeling large amounts of 3D data. At the same time, large 3D repositories s…”
BackgroundMVSNet: Depth Inference for Unstructured Multi-view Stereo
Yao, Y., Luo, Z., Li, S., Fang, T., & Quan, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_47 · Cited by 1,455
Enhanced Computer Vision With Microsoft Kinect Sensor: A Review
Han, J., Shao, L., Xu, D., & Shotton, J. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2265378 · Cited by 1,367
Self-citationDeferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
“In particular, for 3D scanned objects (e.g., via KinectFusion [Izadi et al. 2011; Newcombe et al. 2011] or multi-view stereo reconstructions [Sch¨onberger and Frahm 2016]), where additional ground truth color images are available, they enable learning photo-realistic synthesis of imagery from novel…”
BackgroundSemantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.28 · Cited by 1,209
Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration
Yang, J., Li, H., Campbell, D., & Jia, Y. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2513405 · Cited by 1,170
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37331-2_42 · Cited by 1,161
“Many current vision applications, such as pedestrian tracking, dense SLAM [2], or object detection [1], can be made more robust through the addition of depth information.”
Background3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions
Zeng, A., Song, S., Niessner, M., Fisher, M., Xiao, J., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.29 · Cited by 1,128
Visual Place Recognition: A Survey
Lowry, S., Sünderhauf, N., Newman, P., Leonard, J. J., Cox, D., Corke, P., & Milford, M. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2496823 · Cited by 1,079
“that provide dense depth information as well as image data has spurred the development of dense mapping techniques [70], [126]–[128], [138], [139].”
BackgroundRobotics: Science and Systems VIII
(2013) · The MIT Press eBooks · doi:10.7551/mitpress/9816.001.0001 · Cited by 1,028
FaceWarehouse: A 3D Facial Expression Database for Visual Computing
Cao, C., Weng, Y., Zhou, S., Tong, Y., & Zhou, K. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.249 · Cited by 1,023
Real-time 3D reconstruction at scale using voxel hashing
Nießner, M., Zollhöfer, M., Izadi, S., & Stamminger, M. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508374 · Cited by 1,002
“KinectFusion [Newcombe et al. 2011; Izadi et al. 2011] recently adopted this volumetric method and demonstrated compelling real-time reconstructions using a commodity GPU.”
BackgroundMethodInfluentialSelf-citationDynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
Newcombe, R. A., Fox, D., & Seitz, S. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298631 · Cited by 951
“As in KinectFusion, volumetric TSDF memory limits geometric extent; but there are many solutions to this in the literature.”
MethodInfluentialSelf-citationSLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945
“Estimation using ICP Camera-Model Tracking: In KinectFusion [11], the up-to-date volumetric map is leveraged to compute a view prediction into the previously estimated camera pose, enabling estimation of the live camera using a fast dense iterative closest point (ICP) algorithm [15].”
MethodInfluentialSelf-citationScene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.377 · Cited by 930
“from [15, 21]), then it is possible to refine our inferred camera pose against that model.”
BackgroundMethodSelf-citationDense visual SLAM for RGB-D cameras
Kerl, C., Sturm, J., & Cremers, D. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696650 · Cited by 909
“As KinectFusion does not optimize previous camera poses, there is no possibility to correct accumulated errors in the model.”
BackgroundMethodInfluentialA benchmark for RGB-D visual odometry, 3D reconstruction and SLAM
Handa, A., Whelan, T., McDonald, J., & Davison, A. J. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907054 · Cited by 906
“We evaluate 5 different odometers; (i) DVO [13] which performs full frame alignment between pairs of RGB-D frames making use of a robust cost function to improve robustness to outliers; (ii) FOVIS [14] which uses FAST feature correspondences between pairs of RGB-D frames to resolve camera motion, relying on traditiona…”
BackgroundMethodInfluentialSelf-citationElasticFusion: Dense SLAM Without A Pose Graph
Whelan, T., Leutenegger, S., Salas Moreno, R., Glocker, B., & Davison, A. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.001 · Cited by 835
“Vertices are associated using projective data association [15].”
BackgroundMethodInfluentialSelf-citationNICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Zhu, Z., Peng, S., Larsson, V., Xu, W., Bao, H., Cui, Z., Oswald, M. R., & Pollefeys, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01245 · Cited by 825
3-D Mapping With an RGB-D Camera
Endres, F., Hess, J., Sturm, J., Cremers, D., & Burgard, W. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2279412 · Cited by 820
“KinectFusion [19] is an impressive approach for surface reconstruction based on a voxel grid that contains the truncated signed distance [20] to the surface.”
BackgroundSUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels
Xiao, J., Owens, A., & Torralba, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.458 · Cited by 803
“Instead (Figure 8), we improve the depth map using a Truncated Signed Distance Function (TSDF) [15] to voxelize the space, accumulating the depth map from nearby frames (e.”
MethodCNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction
Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.695 · Cited by 800
“From the point of view of the type of input data being processed, approaches can be classified into either depth camera-based [21, 30, 11] and monocular camera-based [22, 4, 20].”
BackgroundMethodLearning 6D Object Pose Estimation Using 3D Object Coordinates
Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., & Rother, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_35 · Cited by 782
“We used KinectFusion [17,12] to record the external camera parameters for each frame.”
MethodSelf-citationEdge Computing for Autonomous Driving: Opportunities and Challenges
Liu, S., Liu, L., Tang, J., Yu, B., Wang, Y., & Shi, W. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2915983 · Cited by 778
“It takes KinectFusion [35] as the implementation and provided C++, OpenMP, OpenCL, and CUDA versions of key function kernels for heterogeneous hardware.”
MethodPose estimation for augmented reality: a hands-on survey
Marchand, E., Uchiyama, H., & Spindler, F. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2513408 · Cited by 746
“ectFusion [89] was one of the first systems that enables scene reconstruction and consequently camera localization in real-time and in a way compatible with interactive”
InfluentialEnd-to-End Human Pose and Mesh Reconstruction with Transformers
Lin, K., Wang, L., & Liu, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00199 · Cited by 740
“While pioneer works have demonstrated impressive reconstruction using various sensors, such as depth sensors [28, 43] or inertial measurement units [15, 49], researchers are exploring to use a monocular camera setting that is more efficient and convenient.”
BackgroundVisual SLAM algorithms: a survey from 2010 to 2016
Taketomi, T., Uchiyama, H., & Ikeda, S. (2017) · IPSJ Transactions on Computer Vision and Applications · doi:10.1186/s41074-017-0027-2 · Cited by 725
ElasticFusion: Real-time dense SLAM and light source estimation
Whelan, T., Salas-Moreno, R. F., Glocker, B., Davison, A. J., & Leutenegger, S. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916669237 · Cited by 698
“…alternation and effectively per-surface-element-independent filtering by holding the camera pose fixed during the mapping step (Keller et al., 2013; Newcombe et al., 2011a), which has proven to be a highly efficient technique capable of running in real-time even on resource constrained mobile…”
BackgroundMethodInfluentialSelf-citationiMAP: Implicit Mapping and Positioning in Real-Time
Sucar, E., Liu, S., Ortiz, J., & Davison, A. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00617 · Cited by 670
“However, here we work in the ‘dense SLAM’ paradigm pioneered in [18, 17] where a unified dense scene representation is also the basis for camera tracking.”
MethodSelf-citationBundleFusion: Real-Time Globally Consistent 3D Reconstruction Using On-the-Fly Surface Reintegration
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3054739 · Cited by 662
Self-citationVoxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning
Oleynikova, H., Taylor, Z., Fehr, M., Siegwart, R., & Nieto, J. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202315 · Cited by 658
“There are two main methods for integrating information from a sensor data into a TSDF: raycasting [5] and projection mapping [6] [14].”
BackgroundMethodInfluentialComputer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 643
Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis
Dai, A., Qi, C. R., & NieBner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.693 · Cited by 633
“Since the introduction of commodity range sensors such as the Microsoft Kinect, RGB-D scanning has gained a huge momentum in both offline and real-time contexts [26, 3, 28, 42, 4, 7].”
BackgroundKintinuous: Spatially Extended KinectFusion
Whelan, T. J., Kaess, M., Fallon, M. F., Johannsson, H., Leonard, J. J., & McDonald, J. (2012) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 624
Pointwise Convolutional Neural Networks
Hua, B.-S., Tran, M.-K., & Yeung, S.-K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00109 · Cited by 615
Robust reconstruction of indoor scenes
Choi, S., Zhou, Q.-Y., & Koltun, V. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299195 · Cited by 555
“While 3D models of real-world objects can now be created easily [46, 70], the same combination of quality and reliability has yet to be achieved for complete scenes.”
BackgroundMethodRobust odometry estimation for RGB-D cameras
Kerl, C., Sturm, J., & Cremers, D. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631104 · Cited by 542
“Examples of those approaches are Kinect- Fusion [23] .”
BackgroundBillinghurst, M., Clark, A., & Lee, G. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781601989215 · Cited by 529
Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation
Ferstl, D., Reinbacher, C., Ranftl, R., Ruether, M., & Bischof, H. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.127 · Cited by 528
“[14] proposed a method for simultaneous camera localization and depth fusion in real time.”
MethodVizzo, I., Guadagnino, T., Mersch, B., Wiesmann, L., Behley, J., & Stachniss, C. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3236571 · Cited by 524
“In line with prior work [1], [10], [19], [35], we register the deskewed and subsampled scan P̂∗ to the point cloud built so far, i.”
MethodA Survey of Surface Reconstruction from Point Clouds
Berger, M., Tagliasacchi, A., Seversky, L. M., Alliez, P., Guennebaud, G., Levine, J. A., Sharf, A., & Silva, C. T. (2017) · Computer Graphics Forum · doi:10.1111/cgf.12802 · Cited by 509
“Recent works such as Kinect Fusion [NDI ∗ 11] and extensions [NZIS13] demonstrate the practicality of building such online systems.”
BackgroundFusion4D: real-time performance capture of challenging scenes
Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S. R., Kowdle, A., Escolano, S. O., Rhemann, C., Kim, D., Taylor, J., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925969 · Cited by 500
“Following the nonrigid alignment we then fuse the data from the current frame into the reference volumeV as in [Newcombe et al. 2011].”
BackgroundMethodSelf-citationScanning 3D Full Human Bodies Using Kinects
Tong, J., Zhou, J., Liu, L., Pan, Z., & Yan, H. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.56 · Cited by 498
“Using a Kinect and commodity graphics hardware, [7] presented a system for accurate real-time mapping of complex and arbitrary indoor scenes.”
MethodEfficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments
Behley, J., & Stachniss, C. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.016 · Cited by 494
“We furthermore did not see any benefit from a bilateral filtering of the vertex map before the computation of normal estimates that is usually applied with RGB-D data [18].”
BackgroundInfluentialMatsuki, H., Murai, R., Kelly, P. H. J., & Davison, A. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01708 · Cited by 493
“Differentiable Rendering: The classical method for creating a 3D representation was to unproject 2D observations into 3D space and to fuse them via weighted averaging [16, 23].”
Self-citationAutomated image-based tracking and its application in ecology
Dell, A. I., Bender, J. A., Branson, K., Couzin, I. D., de Polavieja, G. G., Noldus, L. P. J. J., Pérez-Escudero, A., Perona, P., Straw, A. D., Wikelski, M., et al. (2014) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2014.05.004 · Cited by 492
Neural Fields in Visual Computing and Beyond
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., sitzmann, V., & Sridhar, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14505 · Cited by 484
Real-time large-scale dense RGB-D SLAM with volumetric fusion
Whelan, T., Kaess, M., Johannsson, H., Fallon, M. F., Leonard, J. J., & McDonald, J. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364914551008 · Cited by 483
“The entire process is very amenable to parallelisation and greatly benefits in execution time from being implemented on a GPU (Newcombe et al. (2011)).”
BackgroundMethodInfluentialChi, H.-L., Kang, S.-C., & Wang, X. (2013) · Automation in Construction · doi:10.1016/j.autcon.2012.12.017 · Cited by 476
“KinÊtre [72] is an application based on Kinect [73] and allows users to scan an arbitrary object andmake it move bymoving their bodies.”
BackgroundA comprehensive survey of augmented reality assembly research
Wang, X., Ong, S. K., & Nee, A. Y. C. (2016) · Advances in Manufacturing · doi:10.1007/s40436-015-0131-4 · Cited by 476
“Reference [160] improves depth capture by combining the Kinect with a high end Nvidia GPU to capture a dense map of the 3D scene that can be exported to a mesh.”
MethodThe Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth
Watson, J., Aodha, O. M., Prisacariu, V. A., Brostow, G., & Firman, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00122 · Cited by 473
BOP: Benchmark for 6D Object Pose Estimation
Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Buch, A. G., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., et al. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_2 · Cited by 470
Video Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
ROSEFusion: random optimization for online dense reconstruction under fast camera motion
Zhang, J., Zhu, C., Zheng, L., & Xu, K. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459676 · Cited by 463
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., & Luiten, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02018 · Cited by 462
“On similar lines, iSDF [28] used implicit representations to efficiently compute signed-distances, compared to [25, 27].”
BackgroundMethodSelf-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera
Ma, F., Cavalheiro, G. V., & Karaman, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793637 · Cited by 460
“php?benchmark=depth_completion Depth sensing is fundamental in a variety of robotic tasks, including obstacle avoidance, 3D mapping [1, 2], and localization [3].”
BackgroundCooperative Heterogeneous Multi-Robot Systems: A Survey
Rizk, Y., Awad, M., & Tunstel, E. W. (2020) · ACM Computing Surveys · doi:10.1145/3303848 · Cited by 449
“Many algorithms have been proposed with varying degrees of computational complexity [51], using a wide range of sensors including cameras [68], acoustic sensors [225], structured light [144], and electromagnetic signals [137].”
BackgroundInLoc: Indoor Visual Localization with Dense Matching and View Synthesis
Taira, H., Okutomi, M., Sattler, T., Cimpoi, M., Pollefeys, M., Sivic, J., Pajdla, T., & Torii, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00752 · Cited by 449
3D Morphable Face Models—Past, Present, and Future
Egger, B., Smith, W. A. P., Tewari, A., Wuhrer, S., Zollhoefer, M., Beeler, T., Bernard, F., Bolkart, T., Kortylewski, A., Romdhani, S., et al. (2020) · ACM Transactions on Graphics · doi:10.1145/3395208 · Cited by 448
“nsor2. The device employs a structured dot pattern, which allows reconstructing depth from a single frame by sacrificing spatial resolution. Resolution may be improved by accumulating several frames [Newcombe et al. 2011]. With the increased resolution and quality of consumer cameras, passive systems have become the m…”
Real-time computer vision with OpenCV
Pulli, K., Baksheev, A., Kornyakov, K., & Eruhimov, V. (2012) · Communications of the ACM · doi:10.1145/2184319.2184337 · Cited by 440
Tretschk, E., Tewari, A., Golyanik, V., Zollhofer, M., Lassner, C., & Theobalt, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01272 · Cited by 432
“Such sensors can been used for 3D reconstruction and completion of rigid environments [52] and non-rigid objects [103, 51, 28, 75].”
MethodA Comprehensive Survey of Visual SLAM Algorithms
Macario Barros, A., Michel, M., Moline, Y., Corre, G., & Carrel, F. (2022) · Robotics · doi:10.3390/robotics11010024 · Cited by 429
“The KinectFusion algorithm [90] was the first algorithm based on an RGB-D sensor to operate in real-time.”
MethodMaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
Runz, M., Buffier, M., & Agapito, L. (2018) · 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2018.00024 · Cited by 415
ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T. A., & Nießner, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.04405 · Cited by 412
“The RGB-D scanning system must be trivial to use, the data processing must be robust and automatic, the semantic annotations must be crowdsourced, and the flow of data through the system must be handled by a tracking server.”
Towards Pervasive Augmented Reality: Context-Awareness in Augmented Reality
Grubert, J., Langlotz, T., Zollmann, S., & Regenbrecht, H. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2543720 · Cited by 407
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Gu, Q., Kuwajerwala, A., Morin, S., Jatavallabhula, K. M., Sen, B., Agarwal, A., Rivera, C., Paul, W., Ellis, K., Chellappa, R., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610243 · Cited by 406
“Closed-vocabulary semantic mapping in 3D. Early works reconstruct the 3D map through online algorithms like simultaneous localization and mapping (SLAM) [1]–[5] or offline methods like structure-from-motion (SfM) [6], [7].”
BackgroundRLBench: The Robot Learning Benchmark & Learning Environment
James, S., Ma, Z., Arrojo, D. R., & Davison, A. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974707 · Cited by 405
“However, if we would like a manipulation system to make use of a SLAM map, it is not currently clear what the best way to represent this map is: whether it be sparse [54], [55], dense [56], [57], or semi-dense [58].”
BackgroundSelf-citationReal-time non-rigid reconstruction using an RGB-D camera
Zollhöfer, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., Loop, C., Theobalt, C., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601165 · Cited by 405
Self-citationVisual SLAM and Structure from Motion in Dynamic Environments: A Survey
Saputra, M. R. U., Markham, A., & Trigoni, N. (2019) · ACM Computing Surveys · doi:10.1145/3177853 · Cited by 398
“, KinectFusion [115] or LSD-SLAM [36], are gaining more popularity.”
BackgroundGS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang, D., & Li, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01853 · Cited by 390
“Traditional SLAM methods employ point/surfel clouds [18, 29, 39, 43], mesh representations [23], voxel hashing [10, 16, 21] or voxel grids [19] as scene representations to construct dense mapping, and have made considerable progress on localization accuracy.”
MethodKinect range sensing: Structured-light versus Time-of-Flight Kinect
Sarbolandi, H., Lefloch, D., & Kolb, A. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.05.006 · Cited by 389
“Even though the camera was mainly designed for gaming, it achieved great popularity in the scientific community where researchers have developed a huge amount of innovative applications that are related to different fields such as online 3D reconstruction [25, 41, 43], medical applications and health care [15, 1], aug…”
BackgroundMethodA Review of Visual-LiDAR Fusion based Simultaneous Localization and Mapping
Debeunne, C., & Vivet, D. (2020) · Sensors · doi:10.3390/s20072068 · Cited by 386
“Nevertheless, this technology is not mature enough to be able to conclude about its performances for SLAM applications.”
BackgroundReal-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion
Keller, M., Lefloch, D., Lambers, M., Izadi, S., Weyrich, T., & Kolb, A. (2013) · 2013 International Conference on 3D Vision · doi:10.1109/3dv.2013.9 · Cited by 384
“To estimate the 6DoF camera pose, the model points are projected from the previous camera pose, and a pyramid-based dense iterative closest point (ICP) [11] alignment is performed using this rendered model map and input depth map.”
MethodSelf-citationIMLS-SLAM: Scan-to-Model Matching Based on 3D Data
Deschaud, J.-E. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460653 · Cited by 383
BAD SLAM: Bundle Adjusted Direct RGB-D SLAM
Schops, T., Sattler, T., & Pollefeys, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00022 · Cited by 383
“Examples include voxel-based KinectFusion-style approaches [28, 45, 47], similar methods based on surfel representations [29, 35, 36], as well as some keyframe-based techniques [39–43].”
BackgroundMethodA survey of state-of-the-art on visual SLAM
Abaspur Kazerouni, I., Fitzgerald, L., Dooly, G., & Toal, D. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117734 · Cited by 383
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Chabra, R., Lenssen, J. E., Ilg, E., Schmidt, T., Straub, J., Lovegrove, S., & Newcombe, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_36 · Cited by 380
Self-citationHuang, M. Q., Ninić, J., & Zhang, Q. B. (2021) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2020.103677 · Cited by 370
6-DoF object pose from semantic keypoints
Pavlakos, G., Zhou, X., Chan, A., Derpanis, K. G., & Daniilidis, K. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989233 · Cited by 366
“More specifically, a complete 3D model of the gas canister was reconstructed using KinectFusion [40] and the groundtruth object pose for each image was calculated by ICP with careful manual initialization.”
MethodSliding Shapes for 3D Object Detection in Depth Images
Song, S., & Xiao, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_41 · Cited by 359
“To make use of self-occlusion, we render the CG model and compute the Truncated Signed Distance Function (TSDF) [14] as a feature.”
MethodThe Replica Dataset: A Digital Replica of Indoor Spaces
Straub, J., Whelan, T. J., Ma, L., Chen, Y., Wijmans, E., Green, S., Engel, J. J., Mur-Artal, R., Ren, C. Y., Verma, S., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05797 · Cited by 357
“Given the 6 DoF poses from the SLAM system, depth images are fused into a truncated signed distance function (TSDF) akin to KinectFusion [23].”
MethodSelf-citationSpace-time Neural Irradiance Fields for Free-Viewpoint Video
Xian, W., Huang, J.-B., Kopf, J., & Kim, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00930 · Cited by 357
“While fusing multiple depth maps into a single, global representation has a long tradition, most work on volumetric depth integration has focused on static scenes [12, 46] or geometry alone without textures [45].”
Background3D Reconstruction with Spatial Memory
Wang, H., & Agapito, L. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00013 · Cited by 356
“To obtain accurate reconstruction, these methods either require a depth/LiDAR sensor [53] or careful initialization and various assumptions about the camera motion and scene appearance [21, 52, 54].”
MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors
Murai, R., Dexheimer, E., & Davison, A. J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01556 · Cited by 350
“…for the relative pose, we can use transform T kf and update the canonical pointmap ˜ X kk via a running weighted average filter [5, 28]: The pointmap initially has larger errors and less confidence due to only using small baseline frames, but filtering merges information from many viewpoints.”
Self-citationRobust real-time visual odometry for dense RGB-D mapping
Whelan, T., Johannsson, H., Kaess, M., Leonard, J. J., & McDonald, J. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631400 · Cited by 344
“The Kintinuous system is an extension of the KinectFusion algorithm published by Newcombe et al. in 2011 [6].”
MethodState of the Art on 3D Reconstruction with RGB‐D Cameras
Zollhöfer, M., Stotko, P., Görlitz, A., Theobalt, C., Nießner, M., Klein, R., & Kolb, A. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13386 · Cited by 339
NICP: Dense normal based point cloud registration
Serafin, J., & Grisetti, G. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353455 · Cited by 335
Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008–2017)
Kim, K., Billinghurst, M., Bruder, G., Duh, H. B.-L., & Welch, G. F. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2868591 · Cited by 335
“For example, Newcombe et al.’s highly cited work on KinectFusion [99], which inspired many researchers developing tracking systems using depth sensors (e.g. [147]).”
BackgroundMethodInfluentialModeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking
Nguyen, C. V., Izadi, S., & Lovell, D. (2012) · 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission · doi:10.1109/3dimpvt.2012.84 · Cited by 334
“Raw depth data was recorded and processed off-line for direct comparison between the original KinectFusion [4, 7] implementation and our extensions.”
MethodSelf-citationSceneNN: A Scene Meshes Dataset with aNNotations
Hua, B.-S., Pham, Q.-H., Nguyen, D. T., Tran, M.-K., Yu, L.-F., & Yeung, S.-K. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.18 · Cited by 333
“KinectFusion [19] and its moving volume extension [33, 23, 20] can be applied to reconstructing a scene in real time.”
BackgroundColored Point Cloud Registration Revisited
Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.25 · Cited by 331
“Dense reconstruction from RGB-D sequences has been extensively studied [29, 17, 24, 10, 4, 41].”
Background
Metadata
- Authors
- Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S.
- Affiliations
- Imperial College London (GB)
- Microsoft (United States) (US)
- Venue
- 2011 10th IEEE International Symposium on Mixed and Augmented Reality
- Publisher
- IEEE
- Published
- 2011-10
- Pages
- 127-136
- ISSN
- 1554-7868 (ISSN-L)
- OpenAlex
- W1987648924
- Cited by
- 4,008
- References
- 34
- Citations by year
- 2026: 120 · 2025: 200 · 2024: 213 · 2023: 252 · 2022: 231 · 2021: 299 · 2020: 314 · 2019: 357 · 2018: 383 · 2017: 361 · 2016: 322 · 2015: 343 · 2014: 256 · 2013: 243 · 2012: 112
- FWCI
- 378.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR (publishedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- KinectFusion, real-time 3D reconstruction, depth camera, Kinect sensor, iterative closest point, pose tracking, indoor scene reconstruction, augmented reality, GPU rendering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.