DTAM: Dense tracking and mapping in real-time
The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Abstract
DTAM is a system for real-time camera tracking and reconstruction which relies not on feature extraction but dense, every pixel methods. As a single hand-held RGB camera flies over a static scene, we estimate detailed textured depth maps at selected keyframes to produce a surface patchwork with millions of vertices. We use the hundreds of images available in a video stream to improve the quality of a simple photometric data term, and minimise a global spatially regularised energy functional in a novel non-convex optimisation framework. Interleaved, we track the camera's 6DOF motion precisely by frame-rate whole image alignment against the entire dense model. Our algorithms are highly parallelisable throughout and DTAM achieves real-time performance using current commodity GPU hardware. We demonstrate that a dense model permits superior tracking performance under rapid motion compared to a state of the art method using features; and also show the additional usefulness of the dense model for real-time scene interaction in a physics-enhanced augmented reality application.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, in our network, multi-scale projections not only help to overcome gradient locality during training (see [33], [47] for a broader discussion of this well-known issue), but also aid in online error correction at runtime.”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [33]
In the citing paper
“SFMLearner [33] demonstrated the unsupervised learning of unscaled egomotion and depth from RGB imagery.”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [33]
In the citing paper
“For KITTI, VIOLearner RGB outperforms approaches SFMLearner [33] and the network of Zhan et al. [30].”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [33]
In the citing paper
“For AirSim, VIOLearner (no IMU) outperforms approaches SFMLearner [33] and ORB-SLAM2-S in translation error.”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [33]
In the citing paper
“Compared to other vision-only approaches, for KITTI, VIOLearner (no IMU) outperforms approaches SFMLearner [33] and the network of Zhan et al. [30] and yet is outperformed by ORB-SLAM-2 which uses full bundle adjustment.”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [33]
Citations
Cites
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging
Chambolle, A., & Pock, T. (2011) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-010-0251-1 · Cited by 4,336
Lucas-Kanade 20 Years On: A Unifying Framework
Baker, S., & Matthews, I. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000011205.11775.fd · Cited by 3,155
Fast cost-volume filtering for visual correspondence and beyond
Rhemann, C., Hosni, A., Bleyer, M., Rother, C., & Gelautz, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995372 · Cited by 485
Live dense reconstruction with a single moving camera
Newcombe, R. A., & Davison, A. J. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539794 · Cited by 366
Self-citationImproving the Agility of Keyframe-Based SLAM
Klein, G., & Murray, D. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88688-4_59 · Cited by 335
Robust multi-sensor image alignment
Irani, M., & Anandan, P. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710832 · Cited by 220
Real-Time Dense Geometry from a Handheld Camera
Stühmer, J., Gumhold, S., & Cremers, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15986-2_2 · Cited by 205
Some First-Order Algorithms for Total Variation Based Image Restoration
Aujol, J. (2009) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-009-0149-y · Cited by 151
Cited by
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778
“Finally, because direct methods are, in general, very computationally demanding, the map is just incrementally expanded as in DTAM, or map optimization is reduced to a pose graph, discarding all sensor measurements as in LSD-SLAM.”
MethodKinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008
Self-citationLSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851
“While direct image alignment is well-established for RGB-D or stereo sensors [14,4], only recently monocular direct VO algorithms have been proposed: In [24,20,21], accurate and fully dense depth maps are computed using a variational formulation, which however is computationally demanding and requires”
MethodPast, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“Right: dense map of a desktop produced by DTAM [184].”
InfluentialORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153
Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Engel, J., Koltun, V., & Cremers, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,924
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910
“Structure from motion The simultaneous estimation of structure and motion is a well studied problem with an established toolchain of techniques [12, 50, 38].”
BackgroundPoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
“Metric SLAM localizes a mobile robot by focusing on creating a sparse [13, 11] or dense [16, 7] map of the environment.”
BackgroundSVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
“With DTAM [15], a novel direct method was introduced that computes a dense depthmap for each keyframe through minimisation of a global, spatially-regularised energy functional .”
MethodKinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047270 · Cited by 2,110
“the Kinect-based system of [12] reconstructs at ∼2Hz), or support real-time camera tracking but non real-time reconstruction or mapping phases [15, 19, 20].”
BackgroundMethodSelf-citationTanks and temples: benchmarking large-scale scene reconstruction
Knapitsch, A., Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073599 · Cited by 1,707
“Direct methods have demonstrated that the camera can be localized and a dense or semi-dense 3D model can be created without sparse feature matching [Engel et al. 2017, 2014; Newcombe et al. 2011].”
MethodUnsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632
“it is a fair 8 Much like the log depth, inverse depth parametrization is less prone to the higher depth errors at very distant points and is used successfully in many stereo [13] and SLAM frameworks [27]. 12 Ravi Garg, Vijay Kumar B G, Gustavo Carneiro, Ian Reid Table 3. Comparison of proposed auto-encoder framework w…”
MethodUnsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., BG, V. K., Carneiro, G., & Reid, I. (2016) · arXiv · doi:10.48550/arxiv.1603.04992 · Cited by 1,632
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00212 · Cited by 1,256
DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes
Bescos, B., Facil, J. M., Civera, J., & Neira, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2860039 · Cited by 1,263
Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., & Navab, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37331-2_42 · Cited by 1,161
Visual Place Recognition: A Survey
Lowry, S., Sünderhauf, N., Newman, P., Leonard, J. J., Cox, D., Corke, P., & Milford, M. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2496823 · Cited by 1,079
“The metric information within the topological place node can be stored as a sparse landmark map [2], [7], [76], or as a dense occupancy grid map [134] if depth information is extracted from the image data.”
BackgroundMethodDeepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks
Wang, S., Clark, R., Wen, H., & Trigoni, N. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989236 · Cited by 970
“Direct methods, in contrast, are capable of exploiting all the pixels in consecutive images for pose estimation under the assumption of photometric consistency, e.g., DTAM in [11].”
MethodScene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.377 · Cited by 930
“This data could be captured in several ways, for example by tracking from depth camera input [15, 21], or by using dense reconstruction and tracking from RGB input [22].”
MethodSLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945
“While this approach is possible with an RGB camera [12], commodity depth cameras have now come to the fore in high performance, robust indoor 3D mapping, in particular via the KinectFusion algorithm [11].”
MethodSelf-citationDense visual SLAM for RGB-D cameras
Kerl, C., Sturm, J., & Cremers, D. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696650 · Cited by 909
“proposed to incrementally build a dense model of the scene and registering every new measurement to this model [5], [23].”
BackgroundSVO: Semidirect Visual Odometry for Monocular and Multicamera Systems
Forster, C., Zhang, Z., Gassner, M., Werlberger, M., & Scaramuzza, D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2623335 · Cited by 922
“When a depth filter’s uncertainty becomes small enough, a new 3D point is inserted in the map and is immediately used for motion estimation.”
BackgroundMethodInfluentialA benchmark for RGB-D visual odometry, 3D reconstruction and SLAM
Handa, A., Whelan, T., McDonald, J., & Davison, A. J. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907054 · Cited by 906
“Multi-view stereo fusion of images has long been a research focus of the vision community, and has recently become feasible in real-time for reconstructing surfaces from monocular video [1], [2].”
BackgroundSelf-citationDynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes
Bescos, B., Fácil, J. M., Civera, J., & Neira, J. (2018) · Zaguan (Universidad de Zaragoza) · Cited by 893
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Zhu, Z., Peng, S., Larsson, V., Xu, W., Bao, H., Cui, Z., Oswald, M. R., & Pollefeys, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01245 · Cited by 825
“One of the early examples of this category was DTAM [29].”
BackgroundElasticFusion: Dense SLAM Without A Pose Graph
Whelan, T., Leutenegger, S., Salas Moreno, R., Glocker, B., & Davison, A. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.001 · Cited by 835
“We adopt an architecture which is typically found in realtime dense visual SLAM systems that alternates between tracking and mapping [15, 25, 9, 8, 2, 16].”
MethodSelf-citationGeometric Loss Functions for Camera Pose Regression with Deep Learning
Kendall, A., & Cipolla, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.694 · Cited by 803
“State of the art localisation systems perform very well within controlled environments [24, 34, 12, 33, 44].”
BackgroundCNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction
Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.695 · Cited by 800
“From the point of view of the type of input data being processed, approaches can be classified into either depth camera-based [21, 30, 11] and monocular camera-based [22, 4, 20].”
BackgroundMethodInfluentialVisual-lidar odometry and mapping: low-drift, robust, and fast
Zhang, J., & Singh, S. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139486 · Cited by 747
Large Scale Multi-view Stereopsis Evaluation
Jensen, R., Dahl, A., Vogiatzis, G., Tola, E., & Aanaes, H. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.59 · Cited by 729
“On the other hand, point-cloud based methods convert photo-consistency data into a 3D pointcloud, which is then converted into a 3D surface using standard meshing techniques such as Poisson reconstruction [9], Graph-cuts [20] or signed distance functions [13].”
MethodPose estimation for augmented reality: a hands-on survey
Marchand, E., Uchiyama, H., & Spindler, F. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2513408 · Cited by 746
“2, dense or direct approaches such as DTAM [34], [90] or [137] allow each pixel contributing to the registration”
Visual SLAM algorithms: a survey from 2010 to 2016
Taketomi, T., Uchiyama, H., & Ikeda, S. (2017) · IPSJ Transactions on Computer Vision and Applications · doi:10.1186/s41074-017-0027-2 · Cited by 725
“Basically, these methods [43, 46, 47] are designed for fast and online 3D modeling.”
MethodInfluentialDeMoN: Depth and Motion Network for Learning Monocular Stereo
Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.596 · Cited by 692
“DTAM [30] can track camera poses reliably for critical motions by matching against dense depth maps.”
BackgroundElasticFusion: Real-time dense SLAM and light source estimation
Whelan, T., Salas-Moreno, R. F., Glocker, B., Davison, A. J., & Leutenegger, S. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916669237 · Cited by 698
“In addition to this, as in previous work we assume tracking operates over a narrow baseline (Newcombe et al. (2011b)).”
BackgroundMethodInfluentialSelf-citationiMAP: Implicit Mapping and Positioning in Real-Time
Sucar, E., Liu, S., Ortiz, J., & Davison, A. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00617 · Cited by 670
“However, here we work in the ‘dense SLAM’ paradigm pioneered in [18, 17] where a unified dense scene representation is also the basis for camera tracking.”
Self-citationLarge-Scale Data for Multiple-View Stereopsis
Aanæs, H., Jensen, R. R., Vogiatzis, G., Tola, E., & Dahl, A. B. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0902-9 · Cited by 672
“…meth-ods convert photo-consistency data into a 3D point-cloud, which is then converted into a 3D triangulated surface using standard meshing techniques such as Poisson reconstruction (Kazhdan et al. 2006), graph cuts (Vogiatzis et al. 2007) or signed distance functions (Newcombe et al. 2011).”
MethodBundleFusion: Real-Time Globally Consistent 3D Reconstruction Using On-the-Fly Surface Reintegration
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3054739 · Cited by 662
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 643
BARF: Bundle-Adjusting Neural Radiance Fields
Lin, C.-H., Ma, W.-C., Torralba, A., & Lucey, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00569 · Cited by 628
“Given a set of input images, S f M [37, 38, 48, 49, 1, 62] and SLAM [33, 13, 32, 64] systems aim to recover the 3D structure and the sensor poses simultaneously.”
BackgroundUnDeepVO: Monocular Visual Odometry Through Unsupervised Deep Learning
Li, R., Wang, S., Long, Z., & Gu, D. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461251 · Cited by 581
Kintinuous: Spatially Extended KinectFusion
Whelan, T. J., Kaess, M., Fallon, M. F., Johannsson, H., Leonard, J. J., & McDonald, J. (2012) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 624
Gallego, G., Delbrück, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.3929/ethz-b-000521357 · Cited by 591
Self-citationPL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments
Gomez-Ojeda, R., Moreno, F.-A., Zuniga-Noel, D., Scaramuzza, D., & Gonzalez-Jimenez, J. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2899783 · Cited by 560
Robust odometry estimation for RGB-D cameras
Kerl, C., Sturm, J., & Cremers, D. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631104 · Cited by 542
“To achieve robustness against outliers many approaches use binary thresholding [21], [24] to segment the image into inliers and outliers, or continuous methods from robust statistics [25], [26].”
MethodModelling uncertainty in deep learning for camera relocalization
Kendall, A., & Cipolla, R. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487679 · Cited by 548
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundGeometry-Aware Learning of Maps for Camera Localization
Brahmbhatt, S., Gu, J., Kim, K., Hays, J., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00277 · Cited by 530
Semi-dense Visual Odometry for a Monocular Camera
Engel, J., Sturm, J., & Cremers, D. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.183 · Cited by 517
“In particular for offline reconstruction, there is a long history of using different baselines to steer the stereo-inherent trade-off between accuracy and precision [12].”
BackgroundDF-Net: Unsupervised Joint Learning of Depth and Flow Using Cross-Task Consistency
Zou, Y., Luo, Z., & Huang, J.-B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_3 · Cited by 469
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., & Luiten, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02018 · Cited by 462
“Traditional approaches to dense SLAM have explored a variety of explicit representations, including 2.5D images [15, 34], (truncated) signed-distance functions [6, 25, 26, 42], gaussian mixture models [9, 10] and circular sur-fels [13, 33, 43].”
BackgroundVideo Based Reconstruction of 3D People Models
Alldieck, T., Magnor, M., Xu, W., Theobalt, C., & Pons-Moll, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00875 · Cited by 468
A survey on vision-based UAV navigation
Lu, Y., Xue, Z., Xia, G.-S., & Zhang, L. (2018) · Geo-spatial Information Science · doi:10.1080/10095020.2017.1420509 · Cited by 448
“Newcombe, Lovegrove, and Davison (2011) presented a real-time monocular SLAM algo-rithm, DTAM, which also estimates the camera’s 6DOF motion using direct methods.”
BackgroundComputing Systems for Autonomous Driving: State of the Art and Challenges
Liu, L., Lu, S., Zhong, R., Wu, B., Yao, Y., Zhang, Q., & Shi, W. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3043716 · Cited by 450
“Direct matching [125]–[127] is called dense visual SLAM, which adopts original information for matching that is dense in the image, such as color and depth from an RGBD camera.”
BackgroundReal-Time 3D Reconstruction and 6-DoF Tracking with an Event Camera
Kim, H., Leutenegger, S., & Davison, A. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_21 · Cited by 438
“Following many recent successful SLAM systems such as PTAM [13], DTAM [20], and LSD-SLAM [10], which separate the tracking and mapping components based on the assumption that the current estimate from one component is accurate enough to lock for the purposes of estimating the other, the basic structure of our approach…”
MethodSelf-citationA Comprehensive Survey of Visual SLAM Algorithms
Macario Barros, A., Michel, M., Moline, Y., Corre, G., & Carrel, F. (2022) · Robotics · doi:10.3390/robotics11010024 · Cited by 429
MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
Runz, M., Buffier, M., & Agapito, L. (2018) · 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2018.00024 · Cited by 415
Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras
Wang, R., Schworer, M., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.421 · Cited by 401
“While the traditional structure and motion/visual SLAM algorithms were based on heuristically selected keypoints, more recently a number of so-called direct methods were proposed [27, 23, 10, 12, 8].”
MethodGS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang, D., & Li, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01853 · Cited by 390
“DTAM [20] first introduces a dense SLAM system that uses photometric consistency to track a handheld camera and represent the scene as a cost volume.”
BackgroundA survey of state-of-the-art on visual SLAM
Abaspur Kazerouni, I., Fitzgerald, L., Dooly, G., & Toal, D. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117734 · Cited by 383
BAD SLAM: Bundle Adjusted Direct RGB-D SLAM
Schops, T., Sattler, T., & Pollefeys, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00022 · Cited by 383
“Direct [10, 45, 46, 69] approaches can make use of all data.”
BackgroundA Review of Visual-LiDAR Fusion based Simultaneous Localization and Mapping
Debeunne, C., & Vivet, D. (2020) · Sensors · doi:10.3390/s20072068 · Cited by 386
“In this category, the most known methods are DTAM [25], LSD-SLAM [26], SVO [27], or DSO [28,29].”
MethodReal-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion
Keller, M., Lefloch, D., Lambers, M., Izadi, S., Weyrich, T., & Kolb, A. (2013) · 2013 International Conference on 3D Vision · doi:10.1109/3dv.2013.9 · Cited by 384
Discrete-Continuous Depth Estimation from a Single Image
Liu, M., Salzmann, M., & He, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.97 · Cited by 381
“Similarly, structure-from-motion [5] and SLAM [19] have now reached the stage where existing systems can efficiently handle huge amounts of images.”
BackgroundCodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM
Bloesch, M., Czarnowski, J., Clark, R., Leutenegger, S., & Davison, A. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00271 · Cited by 372
Self-citationReal-time visual odometry from dense RGB-D images
Steinbrucker, F., Sturm, J., & Cremers, D. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130321 · Cited by 364
Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
Yang, N., Wang, R., Stückler, J., & Cremers, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_50 · Cited by 346
Robust real-time visual odometry for dense RGB-D mapping
Whelan, T., Johannsson, H., Kaess, M., Leonard, J. J., & McDonald, J. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631400 · Cited by 344
“Given the low cost of the sensor coupled with the large scale availability of GPUs for high performance computing, dense methods are becoming more popular in tackling some of the key perception problems in robotics [5], [6], [7].”
BackgroundDROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras
Teed, Z., & Deng, J. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10869 · Cited by 227
“Early work was built using probabilistic and filtering based approaches [12, 30], and alternating optimization of the map and camera poses [34, 16].”
MethodKornia: an Open Source Differentiable Computer Vision Library for PyTorch
Riba, E., Mishkin, D., Ponsa, D., Rublee, E., & Bradski, G. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093363 · Cited by 333
“Multi-view reconstruction is a well understood problem with a good geometric model [21], and many approaches for matching and optimization [39, 8, 51], and some recent promising deep learning approaches [34].”
BackgroundREMODE: Probabilistic, monocular dense reconstruction in real time
Pizzoli, M., Forster, C., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907233 · Cited by 330
“If, on one hand, estimating the depth independently for every pixel leads to efficient, parallel implementations, on the other hand the authors of [10], [11], [12] argued that, similar to other computer vision problems, such as image de-noising [13] and optical flow estimation [14], a smoothing step is required in ord…”
BackgroundMethodInfluentialVision-Based Navigation Techniques for Unmanned Aerial Vehicles: Review and Challenges
Arafat, M. Y., Alam, M. M., & Moh, S. (2023) · Drones · doi:10.3390/drones7020089 · Cited by 319
“Newcombe et al. [85] proposed dense tracking and mapping (DTAM), a real-time monocular SLAM system that uses direct techniques to estimate the 6DOF motion of the camera.”
BackgroundFusion++: Volumetric Object-Level SLAM
Mccormac, J., Clark, R., Bloesch, M., Davison, A., & Leutenegger, S. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00015 · Cited by 308
Self-citationImage Retrieval for Image-Based Localization Revisited
Sattler, T., Weyand, T., Leibe, B., & Kobbelt, L. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.76 · Cited by 304
“With the widespread adoption of mobile camera phones, image-based localization has recently received increased attention as it enables many interesting applications such as mobile landmark recognition [4], real-time camera pose tracking [13, 21, 37], robot navigation [19], or community-based city reconstruction [36].”
BackgroundSimultaneous Optical Flow and Intensity Estimation from an Event Camera
Bardow, P., Davison, A. J., & Leutenegger, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.102 · Cited by 295
“As systems like LSD-SLAM [8] very clearly highlight, pixels where edges move are the only ones which give information useful for tracking and reconstruction, and it is precisely these locations which are highlighted in hardware by an event camera.”
MethodSelf-citationNeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields
Rosinol, A., Leonard, J. J., & Carlone, L. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341922 · Cited by 281
“For instance, DTAM[10], which achieves dense SLAM, follows the same approach as sparse PTAM[11] by first tracking the camera pose and then estimating the depth in a decoupled manner.”
MethodEvery Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding
Luo, C., Yang, Z., Wang, P., Wang, Y., Xu, W., Nevatia, R., & Yuille, A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2930258 · Cited by 301
“e and object motion) and optical flow between consecutive frames from a video are fundamental problems in computer vision. They enable a wide range of real-world applications such as augmented reality [1], video analysis [2], [3] and robotics navigation [4], [5]. In this paper, we propose an effective learning framewor…”
BackgroundMethodInfluentialNeRF--: Neural Radiance Fields Without Known Camera Parameters
Wang, Z., Wu, S., Xie, W., Chen, M., & Prisacariu, V. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07064 · Cited by 245
“In addition to simplifying the original two-stage approach, this joint optimisation approach enables globally consistent reconstruction results, similar to the bundle adjustment used in conventional SfM pipelines [35] and SLAM systems [25, 27, 29].”
MethodResultVery High Frame Rate Volumetric Integration of Depth Images on Mobile Devices
Kahler, O., Adrian Prisacariu, V., Yuheng Ren, C., Sun, X., Torr, P., & Murray, D. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2459891 · Cited by 294
“The insights have also been applied to 3D reconstruction from monocular cameras [13, 17].”
BackgroundCross-View Semantic Segmentation for Sensing Surroundings
Pan, B., Sun, J., Leung, H. Y. T., Andonian, A., & Zhou, B. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3004325 · Cited by 291
“re also integrating multiple local semantic maps into a semantic floor map for further applications. Note that there is a fundamental difference between the cross-view segmentation and the visual SLAM [28, 29]. While both estimate the spatial configuration of an environment, our VPN aims at parsing the top-down-view sem…”
BackgroundRobust monocular SLAM in dynamic environments
Tan, W., Liu, H., Dong, Z., Zhang, G., & Bao, H. (2013) · 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2013.6671781 · Cited by 249
A review of video surveillance systems
Elharrouss, O., Almaadeed, N., & Al-Maadeed, S. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103116 · Cited by 283
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., et al. (2013) · Medical Image Analysis · doi:10.1016/j.media.2013.04.003 · Cited by 280
“Dense Tracking And Matching (DTAM) (Newcombe et al., 2011) builds on this approach to produce dense maps and robust camera localization.”
BackgroundInformation fusion in navigation systems via factor graph based incremental smoothing
Indelman, V., Williams, S., Kaess, M., & Dellaert, F. (2013) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2013.05.001 · Cited by 271
“Several parallel processing data fusion techniques have been introduced recently [28,7,29], including a factor-graph based architecture [30] that utilizes incremental smoothing internally.”
MethodChen, Y., Schmid, C., & Sminchisescu, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00716 · Cited by 261
“Structure from motion (SfM) [11, 17, 34, 37, 39, 44] or scene flow estimation [33, 40] are well-established methodologies with a solid record of fundamental and practical progress.”
BackgroundFurukawa, Y., & Hernández, C. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781601988379 · Cited by 268
Consistency Analysis and Improvement of Vision-aided Inertial Navigation
Hesch, J. A., Kottas, D. G., Bowman, S. L., & Roumeliotis, S. I. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2013.2277549 · Cited by 266
RGB-D SLAM in Dynamic Environments Using Point Correlations
Dai, W., Zhang, Y., Li, P., Fang, Z., & Scherer, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3010942 · Cited by 265
“Factor-graph-optimization-based methods can be further divided into two categories: indirect (feature-based) methods [17], [18], [19] and direct methods [20], [21], [22].”
MethodHinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Konolige, K., Bradski, G., & Navab, N. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33885-4_60 · Cited by 263
BundleFusion: Real-Time Globally Consistent 3D Reconstruction Using On-the-Fly Surface Reintegration
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., & Theobalt, C. (2017) · ACM Transactions on Graphics · doi:10.1145/3054739 · Cited by 258
Feature-Metric Loss for Self-supervised Learning of Depth and Egomotion
Shu, C., Yu, K., Duan, Z., & Yang, K. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58529-7_34 · Cited by 239
ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields
Johari, M. M., Carta, C., & Fleuret, F. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01670 · Cited by 248
“While traditional SLAM systems [16, 41, 45, 55, 76, 77] mostly focus on localization accuracy, recent learningbased dense visual SLAM methods [2, 11, 25, 35, 60, 64, 65, 67, 81, 86] provide meaningful global 3D maps and show reasonable but limited reconstruction accuracy.”
BackgroundConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Gu, Q., Kuwajerwala, A., Morin, S., Jatavallabhula, K. M., Sen, B., Agarwal, A., Rivera, C., Paul, W., Ellis, K., Chellappa, R., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610243 · Cited by 406
A review of visual SLAM methods for autonomous driving vehicles
Cheng, J., Zhang, L., Chen, Q., Hu, X., & Cai, J. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104992 · Cited by 252
Direct Sparse Visual-Inertial Odometry Using Dynamic Marginalization
Von Stumberg, L., Usenko, V., & Cremers, D. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8462905 · Cited by 235
Laser–visual–inertial odometry and mapping with high robustness and low drift
Zhang, J., & Singh, S. (2018) · Journal of Field Robotics · doi:10.1002/rob.21809 · Cited by 250
RIDI: Robust IMU Double Integration
Yan, H., Shan, Q., & Furukawa, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_38 · Cited by 246
DeepTAM: Deep Tracking and Mapping
Zhou, H., Ummenhofer, B., & Brox, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_50 · Cited by 226
Simultaneous Mosaicing and Tracking with an Event Camera
Kim, H., Handa, A., Benosman, R., Ieng, S.-H., & Davison, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.26 · Cited by 247
“We believe these are breakthrough results, showing how joint sequential and global estimation permits the great benefits of an event camera to be applied to a real problem of mosaicing, and hopefully opening the door to similar approaches in dense 3D reconstruction in the style of [16] and many other vision problems.”
BackgroundMethodSelf-citationPoint-SLAM: Dense Neural Point Cloud-based SLAM
Sandström, E., Li, Y., Van Gool, L., & Oswald, M. R. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01690 · Cited by 246
“Like DTAM [38] or BAD-SLAM [50] recent neural SLAM methods [81, 71, 55] only use a single scene representation for both tracking and mapping but they rely either on a regular grid structure [81, 71] or a single MLP [55].”
BackgroundMethodAutonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle
Faessler, M., Fontana, F., Forster, C., Mueggler, E., Pizzoli, M., & Scaramuzza, D. (2016) · Journal of Field Robotics · doi:10.1002/rob.21581 · Cited by 247
“We pe-nalize nonsmooth surfaces by making use of a regularization term based on the Huber norm of the gradient, defined as We chose the Huber norm because it allows smooth reconstruction while preserving discontinuities at strong depth gradient locations (Newcombe et al., 2011).”
MethodInfluential
Metadata
- Authors
- Newcombe, R. A., Lovegrove, S. J., & Davison, A. J.
- Affiliations
- Imperial College London (GB)
- ORCID
- Davison, Andrew J.
- Venue
- 2011 International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2011-11
- Pages
- 2320-2327
- OpenAlex
- W2108134361
- Cited by
- 2,078
- References
- 17
- Citations by year
- 2026: 49 · 2025: 144 · 2024: 163 · 2023: 140 · 2022: 141 · 2021: 166 · 2020: 179 · 2019: 199 · 2018: 183 · 2017: 176 · 2016: 136 · 2015: 151 · 2014: 102 · 2013: 92 · 2012: 51
- FWCI
- 154.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- dense 3D reconstruction, real-time mapping, depth images, non-convex optimization, energy minimization, image registration, GPU parallelization, augmented reality, handheld camera
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.