KinectFusion: Real-time dense surface mapping and tracking
The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Abstract
We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A method for registration of 3-D shapes
Besl, P. J., & McKay, N. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121791 · Cited by 18,103 · cited 2×
“The most important class of algorithms have been based on the ICP concept introduced in [3] which poses data alignment as a nonlinear optimisation problem in which correspondences between scans are approximated using the closest pairs of points found between scans at the previous iteration.”
BackgroundMethodMarching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · Proceedings of the 14th annual conference on Computer graphics and interactive techniques · doi:10.1145/37401.37422 · Cited by 10,104 · cited 1×
“One option is to extract the connected surfaces using a marching cubes type algorithm [18], followed by a standard rasterising rendering pipeline.”
MethodBilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“We apply a bilateral filter [27] to the raw depth map to obtain a discontinuity preserved depth map with reduced noise Dk,”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 3×
“By using only depth data, our system can work in complete darkness mitigating any issues concerning low light conditions, problematic for passive camera [17, 19, 26] and RGB-D based systems [14].”
BackgroundMethodObject modelling by registration of multiple range images
Chen, Y., & Medioni, G. (1992) · Image and Vision Computing · doi:10.1016/0262-8856(92)90066-c · Cited by 3,154
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 6×
“A related non-parametric representation used in graphics is the signed distance function (SDF) introduced in [7] for the purpose of fusing partial depth scans while mitigating problems related to mesh-based reconstruction algorithms.”
BackgroundMethodInfluentialA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · cited 1×
“In computer vision, research on structure from motion (SFM) and multi-view stereo (MVS) has produced many compelling re-sults, in particular accurate camera tracking and sparse reconstructions (e.g. [10]), and increasingly reconstruction of dense surfaces (e.g. [24]).”
BackgroundIterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,402 · cited 1×
“A number of ICP variants perform early iterations on a subset of possibly corresponding points or operate within a coarse-to-fine scheme [31], speeding up both the data association and final pose optimisation.”
BackgroundKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110 · cited 1×
“In [16] we discuss all these possibilities in detail.”
BackgroundSelf-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
Self-citation
Research works printed but not linked: 4
- weise, 0, Inhand scanning with online loop closure, Proceedings of the IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM) 2009
- wurm, 0, OctoMap: A probabilistic, flexible, and compact 3D map representation for robotic systems, Proceedings of the ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation 2010
- 0, NVIDIA OptiX 2 Ray Tracing Engine
- zach, 0, A globally optimal algorithm for robust TV-L1 range image integration, Proceedings of the International Conference on Computer Vision (ICCV) 2007
Cited by
Items 101–200 of 1,006
Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian SplattingPreprint
Yugay, V., Li, Y., Gevers, T., & Oswald, M. R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.10070 · Cited by 330
“A line of works was built upon it improving speed [42] through efficient implementation and volume integration, scalability through voxel hashing [21, 44, 46] and octree data structure [55], and tracking with sparse image features [5] and loop closure [6, 14, 43, 53].”
Background3DMV: Joint 3D-Multi-view Prediction for 3D Semantic Scene Segmentation
Dai, A., & Nießner, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_28 · Cited by 328
Learning to Reconstruct People in Clothing From a Single RGB Camera
Alldieck, T., Magnor, M., Bhatnagar, B. L., Theobalt, C., & Pons-Moll, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00127 · Cited by 326
“DynamicFusion [51] generalized KinectFusion to non-rigid objects by combining non-rigid tracking and fusion.”
BackgroundMethodInfluentialMultiple-Robot Simultaneous Localization and Mapping: A Review
Saeedi, S., Trentini, M., Seto, M., & Li, H. (2016) · Journal of Field Robotics · doi:10.1002/rob.21620 · Cited by 324
“For RGBD and stereo cameras, KinectFusion (Newcombe et al., 2011), ScaViSLAM (Strasdat, Davison, Montiel, & Konolige, 2011), fast odometry (Dryanovski, Valenti, & Xiao, 2013), fast odometry from vision (FOVIS) (Huang et al., 2011), and dense tracking and mapping (Sturm, Bylow, Kahl, & Cremers,…”
MethodAn Overview to Visual Odometry and Visual SLAM: Applications to Mobile Robotics
Yousif, K., Bab-Hadiashar, A., & Hoseinnezhad, R. (2015) · Intelligent Industrial Systems · doi:10.1007/s40903-015-0032-7 · Cited by 322
“Newcombe et al. [61,84] proposed a depth only mapping algorithm using an RGB-D camera.”
MethodUnsupervised feature learning for 3D scene labeling
Lai, K., Bo, L., & Fox, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907298 · Cited by 317
“the output of 3D reconstructions techniques like Kinect Fusion [19] and RGB-D Mapping [20], [21].”
MethodTheory and applications of marker-based augmented reality
Siltanen, S. (2012) · Aaltodoc (Aalto University) · Cited by 317
An overview of depth cameras and range scanners based on time-of-flight technologies
Horaud, R., Hansard, M., Evangelidis, G., & Ménier, C. (2016) · Machine Vision and Applications · doi:10.1007/s00138-016-0784-4 · Cited by 315
“This approach, which is ideally suited to handheld (or robotic) scanning, has been applied to both Kinect v1 [44] and TOF cameras [12].”
MethodEvaluation Strategies for HCI Toolkit Research
Ledo, D., Houben, S., Vermeulen, J., Marquardt, N., Oehlberg, L., & Greenberg, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173610 · Cited by 315
“For instance, 30 fps is often used for realtime tracking systems [79].”
MethodNeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video
Sun, J., Xie, Y., Chen, L., Zhou, X., & Bao, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01534 · Cited by 313
“For other depthbased methods, we use the standard TSDF fusion proposed in [31, 7].”
BackgroundMethodInfluentialRoomAlive: magical experiences enabled by scalable, adaptive projector-camera units
Jones, B., Sodhi, R., Murdock, M., Mehra, R., Benko, H., Wilson, A., Ofek, E., MacIntyre, B., Raghuvanshi, N., & Shapira, L. (2014) · Proceedings of the 27th annual ACM symposium on User interface software and technology · doi:10.1145/2642918.2647383 · Cited by 312
“A rigid transform is estimated between each pair of procam units using a singular value decomposition, and is refined via pairwise iterative closest point [24].”
MethodBigBIRD: A large-scale 3D database of object instances
Singh, A., Sha, J., Narayan, K. S., Achim, T., & Abbeel, P. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906903 · Cited by 311
“Recent work demonstrates that KinectFusion variants can provide high-quality 3D reconstructions [21], [22], [23], [24].”
BackgroundScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans
Dai, A., Ritchie, D., Bokeloh, M., Reed, S., Sturm, J., & Niessner, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00481 · Cited by 311
VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction
Innmann, M., Zollhöfer, M., Nießner, M., Theobalt, C., & Stamminger, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_22 · Cited by 308
Fusion++: Volumetric Object-Level SLAM
Mccormac, J., Clark, R., Bloesch, M., Davison, A., & Leutenegger, S. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00015 · Cited by 308
Self-citationEswaran, M., & Bahubalendruni, M. V. A. R. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.09.016 · Cited by 308
LMSCNet: Lightweight Multiscale 3D Semantic Completion
Roldão, L., de Charette, R., & Verroust-Blondet, A. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00021 · Cited by 301
“First works approximate missing data as a set of primitive local surfaces either from the data structure [37, 35, 11, 37] or using truncated Signed Distance Functions (TSDFs) [8, 30, 34], while others use continuous energy minimization [21].”
RGB-D SLAM in Dynamic Environments Using Static Point Weighting
Li, S., & Lee, D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2724759 · Cited by 298
“The second group is correspondence based method [2], [14]–[16].”
MethodRegistration with the Point Cloud Library: A Modular Framework for Aligning in 3-D
Holz, D., Ichim, A. E., Tombari, F., Rusu, R. B., & Behnke, S. (2015) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2015.2432331 · Cited by 296
“This is the working assumption of cloud registration algorithms deployed within SLAM and Kinect Fusion-like frameworks [29].”
MethodVery High Frame Rate Volumetric Integration of Depth Images on Mobile Devices
Kahler, O., Adrian Prisacariu, V., Yuheng Ren, C., Sun, X., Torr, P., & Murray, D. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2459891 · Cited by 294
“Later the advent of inexpensive depth cameras like the Microsoft Kinect and massively parallel processors in GPUs led to the seminal Kinect Fusion system [14, 8], which has been reimplemented a number of times [20, 1, 21] and inspired a wide range of further work [6, 12, 5].”
BackgroundMethodResultInfluentialNICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM
Zhu, Z., Peng, S., Larsson, V., Cui, Z., Oswald, M. R., Geiger, A., & Pollefeys, M. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00096 · Cited by 295
“Another line of research considers world-centric maps, and they anchor the 3D geometry of a full scene in uniform world coordinates and represent as surfels [63, 47] or occupancies/TSDF values in the voxel grids [3, 10, 37, 34].”
BackgroundDense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
Florence, P., Manuelli, L., & Tedrake, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08756 · Cited by 142
“An alternative route to collecting data which does not require a calibrated robot is to use a dense SLAM method (for example, [28, 29]).”
MethodGe, L., Liang, H., Yuan, J., & Thalmann, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.602 · Cited by 291
“KinectFusion [14] applies the Truncated Signed Distance Function (TSDF) based volumetric representation for environment mapping and localization with depth camera.”
MethodCross-View Semantic Segmentation for Sensing Surroundings
Pan, B., Sun, J., Leung, H. Y. T., Andonian, A., & Zhou, B. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3004325 · Cited by 291
“re also integrating multiple local semantic maps into a semantic floor map for further applications. Note that there is a fundamental difference between the cross-view segmentation and the visual SLAM [28, 29]. While both estimate the spatial configuration of an environment, our VPN aims at parsing the top-down-view sem…”
BackgroundGO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction
Zhang, Y., Tosi, F., Mattoccia, S., & Poggi, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00345 · Cited by 293
“…the SDF for accurate surface reconstructions, we approximate the ground truth SDF of sampling point x i by computing its distance to the keyframe’s [26, 6, 39, 11, 41] and RGB-D SLAM [28, 32, 26, 35, 53] are taken from [41] and [53] respectively. depth D m , i.e ., b ( x i ) = D m − D ,i .”
BackgroundMethodNeural RGB-D Surface Reconstruction
Azinovic, D., Martin-Brualla, R., Goldman, D. B., Niebner, M., & Thies, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00619 · Cited by 288
“, KinectFusion [50] combines such representation with real-time tracking to reconstruct objects and small scenes in real-time.”
BackgroundRIO: 3D Object Instance Re-Localization in Changing Indoor Environments
Wald, J., Avetisyan, A., Navab, N., Tombari, F., & Nießner, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00775 · Cited by 288
“In particular, the rapid progress in RGB-D scanning systems [17, 18, 31, 6] allows to obtain 3D reconstructions of indoor scenes using only low-cost scanning devices such as the Microsoft Kinect, Intel Real Sense, or Google Tango.”
MethodCo-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM
Wang, H., Wang, J., & Agapito, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01277 · Cited by 287
“Taking advantage of commodity depth sensors, KinectFusion [18] performs frame-to-model”
MethodTossingBot: Learning to Throw Arbitrary Objects With Residual Physics
Zeng, A., Song, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.2988642 · Cited by 282
CT-ICP: Real-time Elastic LiDAR Odometry with Loop Closure
Dellenbach, P., Deschaud, J.-E., Jacquet, B., & Goulette, F. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9811849 · Cited by 280
Latent-Class Hough Forests for 3D Object Detection and Pose Estimation
Tejani, A., Tang, D., Kouskouridas, R., & Kim, T.-K. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_30 · Cited by 280
“Whilst previously the requirement of 3D models may have been a disadvantage, with recent innovations in surface reconstruction techniques [54, 82] these can now be obtained easily and efficiently using hand-held RGB-D cameras.”
MethodReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals
Palazzolo, E., Behley, J., Lottes, P., Giguère, P., & Stachniss, C. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967590 · Cited by 279
Immersive Group-to-Group Telepresence
Beck, S., Kunert, A., Kulik, A., & Froehlich, B. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.33 · Cited by 277
“Newcombe, Izadi and colleagues [30, 16] presented a 3D scanning system that uses a single Kinect.”
Background3DRegNet: A Deep Neural Network for 3D Point Registration
Pais, G. D., Ramalingam, S., Govindu, V. M., Nascimento, J. C., Chellappa, R., & Miraldo, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00722 · Cited by 268
BOP Challenge 2020 on 6D Object Localization
Hodaň, T., Sundermeyer, M., Drost, B., Labbé, Y., Brachmann, E., Michel, F., Rother, C., & Matas, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-66096-3_39 · Cited by 268
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Konolige, K., Bradski, G., & Navab, N. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33885-4_60 · Cited by 263
Comparison of Kinect V1 and V2 Depth Images in Terms of Accuracy and Precision
Wasenmüller, O., & Stricker, D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54427-4_3 · Cited by 262
“This considerably pushed forward several research fields such as: 3D reconstruction [1, 2], camera localization and mapping (SLAM) [3, 4], gesture and object recognition [5, 6], bilateral filtering [7, 8], and many more.”
BackgroundNeural-Guided RANSAC: Learning Where to Sample Model Hypotheses
Brachmann, E., & Rother, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00442 · Cited by 260
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving
Wei, Y., Zhao, L., Zheng, W., Zhu, Z., Zhou, J., & Lu, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01986 · Cited by 260
Low-Cost 3D Systems: Suitable Tools for Plant Phenotyping
Paulus, S., Behmann, J., Mahlein, A.-K., Plümer, L., & Kuhlmann, H. (2014) · Sensors · doi:10.3390/s140203001 · Cited by 259
“Various approaches to extract a consistent 3D model out of the depth video stream were developed [16,32,33].”
MethodBundleFusion: Real-Time Globally Consistent 3D Reconstruction Using On-the-Fly Surface Reintegration
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3054739 · Cited by 258
Self-citationRapyuta: A Cloud Robotics Platform
Mohanarajah, G., Hunziker, D., D'Andrea, R., & Waibel, M. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2329556 · Cited by 255
“This demonstration used the point cloud library's (PCL) [33] implementation of the KinectFusion [34] algorithm, which does not rely on keyframes as in the previous demonstration, but utilizes all images and all of their pixels to reconstruct the map.”
MethodVConv-DAE: Deep Volumetric Shape Learning Without Object Labels
Sharma, A., Grau, O., & Fritz, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_20 · Cited by 254
“To overcome some of these difficulties, there is a large body of work on fusing multiple scans into a single 3D model [2].”
BackgroundA Large Dataset of Object Scans
Choi, S., Zhou, Q.-Y., Miller, S., & Koltun, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02481 · Cited by 101
“where np is the surface normal at p and K is a set of corresponding point pairs found by projective data association [5].”
BackgroundRecent advances in connected and automated vehicles
Elliott, D., Keen, W., & Miao, L. (2019) · Journal of Traffic and Transportation Engineering (English Edition) · doi:10.1016/j.jtte.2018.09.005 · Cited by 253
Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation and Gauss-Newton Refinement
Yu, Z., & Gao, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00202 · Cited by 253
A review of visual SLAM methods for autonomous driving vehicles
Cheng, J., Zhang, L., Chen, Q., Hu, X., & Cai, J. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104992 · Cited by 252
State of the Art in Surface Reconstruction from Point Clouds
Berger, M., Tagliasacchi, A., Seversky, L. M., Alliez, P., Levine, J. A., Sharf, A., & Silva, C. T. (2014) · doi:10.2312/egst.20141040 · Cited by 251
VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.284 · Cited by 249
“Ground truth camera poses for the dataset are obtained using the KinectFusion algorithm [13] to produce smooth camera tracks and a dense 3D model of each scene.”
MethodVoxFormer: Sparse Voxel Transformer for Camera-Based 3D Semantic Scene Completion
Li, Y., Yu, Z., Choy, C., Xiao, C., Alvarez, J. M., Fidler, S., Feng, C., & Anandkumar, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00877 · Cited by 249
Robust monocular SLAM in dynamic environments
Tan, W., Liu, H., Dong, Z., Zhang, G., & Bao, H. (2013) · 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2013.6671781 · Cited by 249
“The KinectFusion system [24] fuses the captured depth maps into a scene model maintained with a volumetric, truncated signed distance function representation, and uses multi-scale iterative closest point alignment between the predicted surface and current captured depth map to track camera motion.”
MethodESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields
Johari, M. M., Carta, C., & Fleuret, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01670 · Cited by 248
“Given a set of sequential RGB-D frames { I i , D i } Mi =1 , our model predicts camera poses { R i | t i } Mi =1 and an implicit TSDF ϕ g representation that can be used in marching cubes algo-rithm [33] to extract 3D meshes.”
InfluentialFIESTA: Fast Incremental Euclidean Distance Fields for Online Motion Planning of Aerial Robots
Han, L., Gao, F., Zhou, B., & Shen, S. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968199 · Cited by 248
“Representative mapping frameworks include Octomap [2] and TSDF(Truncated Signed Distance Field) [3], etc.”
Rotation-Invariant Transformer for Point Cloud Matching
Yu, H., Qin, Z., Hou, J., Saleh, M., Li, D., Busam, B., & Ilić, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00521 · Cited by 248
“…correspondence estimation between a pair of partially-overlapping point clouds is a long-standing task that lies at the core of many computer vision applications, such as tracking [17, 18], reconstruction [22, 23, 39], pose estimation [19, 27, 50] and 3D representation learning [15, 16, 46], etc.”
BackgroundScalable real-time volumetric surface reconstruction
Chen, J., Bautembach, D., & Izadi, S. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461940 · Cited by 247
Self-citationPoint-SLAM: Dense Neural Point Cloud-based SLAM
Sandström, E., Li, Y., Van Gool, L., & Oswald, M. R. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01690 · Cited by 246
“The grid-based representation is perhaps the most explored one and can be further split into methods using dense grids [79, 36, 63, 64, 13, 54, 3, 24, 11, 77, 76, 66, 81], hierarchical oc-trees [69, 49, 29, 6, 26] and voxel hashing [38, 21, 15, 60, 33] to save memory.”
BackgroundMethodInfluentialColor map optimization for 3D reconstruction with consumer depth cameras
Zhou, Q.-Y., & Koltun, V. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601134 · Cited by 244
“The resolution of this initial mesh is determined by the resolution of the voxel grid used by the KinectFusion pipeline.”
BackgroundMethodOccuSeg: Occupancy-Aware 3D Instance Segmentation
Han, L., Zheng, T., Xu, L., & Fang, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00301 · Cited by 243
Neural Topological SLAM for Visual Navigation
Singh Chaplot, D., Salakhutdinov, R., Gupta, A., & Gupta, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01289 · Cited by 241
“Researchers have used explicit metric spatial representations [12], and have considered how can such representations be built with different sensors [19,26–28,38], and how can agents be localized against such representations [11].”
BackgroundCalibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors
Pagliari, D., & Pinto, L. (2015) · Sensors · doi:10.3390/s151127569 · Cited by 241
“These libraries create a single shaded 3D model, by integrating and merging depth data, while tracking the sensor pose [19,37].”
Feng, L., Chen, S., Zhang, C., Zhang, Y., & He, Y. (2021) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2021.106033 · Cited by 240
User-defined gestures for augmented reality
Piumsomboon, T., Clark, A., Billinghurst, M., & Cockburn, A. (2013) · CHI '13 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/2468356.2468527 · Cited by 239
“Modern vision-based tracking algorithms can robustly register the environment without markers, allowing for higher mobility [16].”
Autonomous aerial navigation using monocular visual‐inertial fusion
Lin, Y., Gao, F., Qin, T., Gao, W., Liu, T., Wu, W., Yang, Z., & Shen, S. (2018) · Journal of Field Robotics · doi:10.1002/rob.21732 · Cited by 237
iSDF: Real-Time Neural Signed Distance Fields for Robot Perception
Ortiz, J., Clegg, A., Dong, J., Sucar, E., Novotný, D., Zollhöfer, M., & Mukadam, M. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.012 · Cited by 238
SpatialTracker: Tracking Any 2D Pixels in 3D Space
Xiao, Y., Wang, Q., Zhang, S., Xue, N., Peng, S., Shen, Y., & Zhou, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01929 · Cited by 237
“Scene flow estimation is also often solved as a sub-task in non-rigid reconstruction pipelines [31, 35, 43].”
BackgroundVox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation
Yang, X., Li, H., Zhai, H., Ming, Y., Liu, Y., & Zhang, G. (2022) · 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar55827.2022.00066 · Cited by 234
“These poses are then coupled with their corresponding input point clouds to update a global map represented by geometric primitives such as cost volumes [20, 37], surfels [29, 35, 38] or voxels [12, 19, 21].”
BackgroundScan2CAD: Learning CAD Model Alignment in RGB-D Scans
Avetisyan, A., Dahnert, M., Dai, A., Savva, M., Chang, A. X., & NieBner, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00272 · Cited by 231
“Many modern real-time reconstruction methods, such as KinectFusion [18, 29], are based on this surface representation.”
BackgroundMethodLiveCap: Real-Time Human Performance Capture From Monocular Video
Habermann, M., Xu, W., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2019) · ACM Transactions on Graphics · doi:10.1145/3311970 · Cited by 230
“Recent approaches that are based on a single depth camera, such as KinectFusion, enable the reconstruction of 3D rigid scenes [Izadi et al. 2011; Newcombe et al. 2011] and also appearance models [Zhou and Koltun 2014] by incrementally fusing geometry in a canonical frame.”
MethodInfluentialDense scene reconstruction with points of interest
Zhou, Q.-Y., & Koltun, V. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461919 · Cited by 229
ConceptFusion: Open-set multimodal 3D mapping
Jatavallabhula, K., Kuwajerwala, A., Gu, Q., Omama, M., Iyer, G., Saryazdi, S., Chen, T., Maalouf, A., Li, S., Keetha, N., et al. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.066 · Cited by 228
ConceptFusion: Open-set Multimodal 3D Mapping
Jatavallabhula, K. M., Kuwajerwala, A., Gu, Q., Omama, M., Chen, T., Maalouf, A., Li, S., Iyer, G., Saryazdi, S., Keetha, N., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.07241 · Cited by 228
“At the core of nearly all of these methods is the volumetric fusion technique from Curless and Levoy [28], which has been adapted for real-time incremental capture of surface geometry and color [29, 30, 31, 32, 33].”
MethodCo-fusion: Real-time segmentation, tracking and fusion of multiple objects
Runz, M., & Agapito, L. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989518 · Cited by 227
“Successors to KinectFusion have quickly addressed some of its shortcomings.”
BackgroundInfluentialDeformable Shape Completion with Graph Convolutional Autoencoders
Litany, O., Bronstein, A., Bronstein, M., & Makadia, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00202 · Cited by 226
3D-MPA: Multi-Proposal Aggregation for 3D Semantic Instance Segmentation
Engelmann, F., Bokeloh, M., Fathi, A., Leibe, B., & NieBner, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00905 · Cited by 221
“With the availability of commodity RGB-D sensors such as Kinect or Intel RealSense, the computer vision and graphics communities have achieved impressive results on 3D reconstruction methods [27, 28] that can now even achieve global pose tracking in real time [8, 47].”
BackgroundAdditive Manufacturing: Status and Opportunities
Scott, J. A., Weber, C. L., Newsome, S., & Wohlers, T. T. (2012) · Cited by 221
Near‐Eye Display and Tracking Technologies for Virtual and Augmented Reality
Koulieris, G. A., Akşit, K., Stengel, M., Mantiuk, R. K., Mania, K., & Richardt, C. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13654 · Cited by 220
Direct visual-inertial odometry with stereo cameras
Usenko, V., Engel, J., Stuckler, J., & Cremers, D. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487335 · Cited by 219
“Since then, direct methods have been omni-present in the domain of RGB-D cameras [18] [27], as they directly provide the required pixel-wise depth as sensor measurement.”
MethodVisual Camera Re-Localization from RGB and RGB-D Images Using DSAC
Brachmann, E., & Rother, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3070754 · Cited by 217
MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM
Xu, B., Li, W., Tzoumanikas, D., Bloesch, M., Davison, A., & Leutenegger, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794371 · Cited by 217
“It started from sparse SLAM [1], [2], and evolved into dense SLAM [3] thanks to the increased computational power of GPU and affordability of depth sensors.”
BackgroundMethodSelf-citationSLAM; definition and evolution
Taheri, H., & Xia, Z. C. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.104032 · Cited by 217
You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors
Wang, H., Liu, Y., Dong, Z., & Wang, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548023 · Cited by 216
“Finding an accurate SE(3) transformation of two unaligned partial point clouds, known as point cloud registration, is a prerequisite for many tasks such as 3D reconstruction [28, 46, 54, 85], pose estimation [2, 30, 45, 82], AR/VR [29, 50, 71, 83], and autonomous driving [8, 53, 74, 80].”
BackgroundCvxNet: Learnable Convex Decomposition
Deng, B., Genova, K., Yazdani, S., Bouaziz, S., Hinton, G., & Tagliasacchi, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00011 · Cited by 215
“In computer vision and robotics, analogous trade-offs exist: surface models are essential for the construction of low-dimensional parametric templates essential for tracking [6, 8], volumetric representations are key to capturing 3D data whose topology is unknown [48, 47], while part-based models provide a natural dec…”
BackgroundIndoor scene reconstruction using feature sensitive primitive extraction and graph-cut
Oesau, S., Lafarge, F., & Alliez, P. (2014) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2014.02.004 · Cited by 214
“Newcombe et al. (2011) and Izadi et al. (2011) propose a surface reconstruction method from point clouds acquired by a low cost consumer-grade hand held scanner.”
Point-plane SLAM for hand-held 3D sensors
Taguchi, Y., Jian, Y.-D., Ramalingam, S., & Feng, C. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631318 · Cited by 214
“KinectFusion [21] extended conventional ICP algorithms by registering a current depth map to a virtual”
MethodKinect and RGBD Images: Challenges and Applications
Cruz, L., Lucio, D., & Velho, L. (2012) · 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials · doi:10.1109/sibgrapi-t.2012.13 · Cited by 213
Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition
Guo, C., Jiang, T., Chen, X., Song, J., & Hilliges, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01236 · Cited by 214
Self-citationStaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments
Scona, R., Jaimez, M., Petillot, Y. R., Fallon, M., & Cremers, D. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460681 · Cited by 212
“A seminal work in this field is KinectFusion [2], the first system which used RGB-D data to perform real-time dense tracking and data fusion.”
MethodLachat, E., Macher, H., Landes, T., & Grussenmeyer, P. (2015) · Remote Sensing · doi:10.3390/rs71013070 · Cited by 212
“1 mm were reported for the Kinect Fusion model .”
MethodInfluentialAn Overview on Visual SLAM: From Tradition to Semantic
Chen, W., Shang, G., Ji, A., Zhou, C., Wang, X., Xu, C., Li, Z., & Hu, K. (2022) · Remote Sensing · doi:10.3390/rs14133010 · Cited by 211
Event-Based, 6-DOF Camera Tracking from Photometric Depth Maps
Gallego, G., Lund, J. E. A., Mueggler, E., Rebecq, H., Delbruck, T., & Scaramuzza, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2769655 · Cited by 211
“, DTAM [30] or REMODE [31]), RGB-D sensors [32], or even using an event camera (future research).”
BackgroundFourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time
Wang, L., Zhang, J., Liu, X., Zhao, F., Zhang, Y., Zhang, Y., Wu, M., Yu, J., & Xu, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01316 · Cited by 212
FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow
Zhang, T., Zhang, H., Li, Y., Nakamura, Y., & Zhang, L. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197349 · Cited by 209
“For human living environments, benefit from the economic RGB-D sensors and the computational power improvement from economical graphics processing units(GPU), the dense RGB-D fusion based approaches, e.g. , KinectFusion [2] and ElasticFusion(EF) [3], made real-time static indoor environments reconstruction come true wi…”
BackgroundMulti-resolution surfel maps for efficient dense 3D modeling and tracking
Stückler, J., & Behnke, S. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2013.02.008 · Cited by 209
“Newcombe et al. [16] proposed DTAM, an impressive method for dense tracking and mapping of small workspaces that is real-time capable on GPU.”
MethodSDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Jiang, Y., Ji, D., Han, Z., & Zwicker, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00133 · Cited by 208
Orbeez-SLAM: A Real-time Monocular Visual SLAM with ORB Features and NeRF-realized Mapping
Chung, C.-M., Tseng, Y.-C., Hsu, Y.-C., Shi, X.-Q., Hua, Y.-H., Yeh, J.-F., Chen, W., Chen, Y., & Hsu, W. H. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160950 · Cited by 212
“For instance, some studies [19], [20] leverage the SDF to construct the map.”
Vision-Only Robot Navigation in a Neural Radiance World
Adamkiewicz, M., Chen, T., Caccavale, A., Gardner, R., Culbertson, P., Bohg, J., & Schwager, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3150497 · Cited by 206
“This formalism has received particular interest as a map representation following the success of KinectFusion [22], which constructs truncated SDFs using RGB-D data.”
MethodPanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things
Narita, G., Seno, T., Ishikawa, T., & Kaji, Y. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967890 · Cited by 206
Detailed Human Avatars from Monocular Video
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00022 · Cited by 206
Real-Time Camera Tracking and 3D Reconstruction Using Signed Distance Functions
Bylow, E., Sturm, J., Kerl, C., Kahl, F., & Cremers, D. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.035 · Cited by 205
“The linear weight, as proposed by Curless and Levoy [7] and used in KinectFusion, assigns a constant weight to all voxels up to a certain penetration depth .”
BackgroundMethodInfluentialChen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12567 · Cited by 209
“DA-RNN [108] introduces recurrent models into semantic segmentation framework to learn the temporal connections over multiple view frames, producing more accurate and consistent semantic labelling for volumetric maps from KinectFusion [127].”
Method
Metadata
- Authors
- Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S.
- Affiliations
- Imperial College London (GB)
- Microsoft (United States) (US)
- Venue
- 2011 10th IEEE International Symposium on Mixed and Augmented Reality
- Publisher
- IEEE
- Published
- 2011-10
- Pages
- 127-136
- ISSN
- 1554-7868 (ISSN-L)
- OpenAlex
- W1987648924
- Cited by
- 4,008
- References
- 34
- Citations by year
- 2026: 120 · 2025: 200 · 2024: 213 · 2023: 252 · 2022: 231 · 2021: 299 · 2020: 314 · 2019: 357 · 2018: 383 · 2017: 361 · 2016: 322 · 2015: 343 · 2014: 256 · 2013: 243 · 2012: 112
- FWCI
- 378.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR (publishedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- KinectFusion, real-time 3D reconstruction, depth camera, Kinect sensor, iterative closest point, pose tracking, indoor scene reconstruction, augmented reality, GPU rendering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.