The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
DeepVO: Towards end-to-end visual odometry with deep Recurrent Convolutional Neural Networks
Open accessAbstract
This paper studies monocular visual odometry (VO) problem. Most of existing VO algorithms are developed under a standard pipeline including feature extraction, feature matching, motion estimation, local optimisation, etc. Although some of them have demonstrated superior performance, they usually need to be carefully designed and specifically fine-tuned to work well in different environments. Some prior knowledge is also required to recover an absolute scale for monocular VO. This paper presents a novel end-to-end framework for monocular VO by using deep Recurrent Convolutional Neural Networks (RCNNs). Since it is trained and deployed in an end-to-end manner, it infers poses directly from a sequence of raw RGB images (videos) without adopting any module in the conventional VO pipeline. Based on the RCNNs, it not only automatically learns effective feature representation for the VO problem through Convolutional Neural Networks, but also implicitly models sequential dynamics and relations using deep Recurrent Neural Networks. Extensive experiments on the KITTI VO dataset show competitive performance to state-of-the-art methods, verifying that the end-to-end Deep Learning technique can be a viable complement to the traditional VO systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“More surprisingly, they show that the welltrained networks enable scale-consistent predictions over long videos, while the accuracy is still inferior to traditional methods because of ignoring geometric information.”
From DF-VO: What Should Be Learnt for Visual Odometry? (2021) · cites this paper as Wang et al. 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On the opposing side, end-to-end approaches can estimate the 6 degrees of freedom (DoF) poses directly from a sequence of RGB images, that is, it does not need engineering effort to design the modules in the traditional VO pipeline [4].”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“This problem is called monocular visual odometry and often relies on geometric approaches that require considerable engineering effort for a specific scenario.”
From Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach (2023) · cites this paper as [4]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“the amount of crater detections that ODN-detector produces is robust to disturbances or modifications of the image quality due to Gaussian noise, shot noise, brightness levels, as well as different illumination conditions.”
From Optical navigation for Lunar landing based on Convolutional Neural Network crater detector (2022) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“There have been some popular and powerful DNN architectures, such as VGGNet [22] and GoogLeNet [23], developed for computer vision tasks, producing remarkable performance.”
MethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 2×
“In this section, we evaluate the proposed end-to-end monocular VO approach on the well-known KITTI VO/SLAM benchmark [3].”
BackgroundMethod(2016) · Cited by 11,679
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · cited 1×
“Examples include keyframe base PTAM [9] and ORB-SLAM [10].”
BackgroundParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 1×
“Examples include keyframe base PTAM [9] and ORB-SLAM [10].”
BackgroundFlownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · cited 2×
“In order to reduce both the training time and data required to converge, the CNN is based on a pre-trained FlowNet model [24].”
MethodMonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · cited 1×
“To mitigate this problem, visual Simultaneous Localisation and Mapping (SLAM) or Structure from Motion (SfM) can be adopted to maintain a feature map for drift correction along with pose estimation [8].”
MethodORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153
Multiple view geometry in computer vision
Opower, H. (2002) · Optics and Lasers in Engineering · doi:10.1016/s0143-8166(01)00145-2 · Cited by 3,044
Research works printed but not linked: 1
- zaremba, 2014, Learning to execute
Cited by
Sorted by citations
A survey of uncertainty in deep neural networks
Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10562-9 · Cited by 1,670
“[207], [330], [331] developed deep learning based localization systems along with uncertainty estimates.”
BackgroundZhan, H., Garg, R., Weerasekera, C. S., Li, K., Agarwal, H., & Reid, I. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00043 · Cited by 705
UnDeepVO: Monocular Visual Odometry Through Unsupervised Deep Learning
Li, R., Wang, S., Long, Z., & Gu, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461251 · Cited by 596
Self-citationDeep Learning Sensor Fusion for Autonomous Vehicle Perception and Localization: A Review
Fayyad, J., Jaradat, M. A., Gruyer, D., & Najjaran, H. (2020) · Sensors · doi:10.3390/s20154220 · Cited by 559
“To demonstrate the advantage of adopting a recurrent neural network for VO, Wang et al. [154] implemented an end-to-end deep recurrent convolutional network that takes sequential RGB images and detects poses.”
CubeSLAM: Monocular 3-D Object SLAM
Yang, S., & Scherer, S. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2909168 · Cited by 511
“Recently, there is also some end-to-end deep learning-based SLAM without object representations, such as DVSO [28], DeepVO [29].”
BackgroundDeep learning for monocular depth estimation: A review
Ming, Y., Meng, X., Fan, C., & Yu, H. L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.089 · Cited by 490
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438
“End-to-end learned deep neural networks have been explored to directly predict the relative poses between images with supervised [70, 75, 85] or unsupervised learning [46, 73, 82, 86].”
BackgroundDeepIM: Deep Iterative Matching for 6D Pose Estimation
Li, Y., Wang, G., Ji, X., Xiang, Y., & Fox, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_42 · Cited by 576
Visual SLAM and Structure from Motion in Dynamic Environments
Saputra, M. R. U., Markham, A., & Trigoni, N. (2018) · ACM Computing Surveys · doi:10.1145/3177853 · Cited by 401
“[171] propose “DeepVO” as an end-to-end learning framework capable of learning sequential motion dynamics from image sequences through a Recurrent Convolutional Neural Network (RCNN), a combination of CNN and Recurrent Neural Network (RNN).”
BackgroundSelf-citationA survey of state-of-the-art on visual SLAM
Kazerouni, I. A., Fitzgerald, L., Dooly, G., & Toal, D. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117734 · Cited by 420
Niroui, F., Zhang, K., Kashino, Z., & Nejat, G. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2891991 · Cited by 389
“Traditional machine learning and deep learning techniques have been applied to image classification [13]–[16], target object detection [17], [18], robot-team work division [18], and robot visual odometry [19], [20] problems.”
MethodCodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM
Bloesch, M., Czarnowski, J., Clark, R., Leutenegger, S., & Davison, A. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00271 · Cited by 401
Self-citationIONet: Learning to Cure the Curse of Drift in Inertial Odometry
Chen, C., Lu, C. X., Markham, A., & Trigoni, N. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12102 · Cited by 374
Self-citationDeep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
Yang, N., Wang, R., Stückler, J., & Cremers, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_50 · Cited by 346
DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D CamerasPreprint
Teed, Z., & Deng, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10869 · Cited by 360
“Other work has tried to learn SLAM or VO systems end-to-end [59, 47, 53, 52, 46].”
BackgroundAnomaly Detection for Time Series Using VAE-LSTM Hybrid Model
Lin, S., Clark, R., Birke, R., Schönborn, S., Trigoni, N., & Roberts, S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053558 · Cited by 360
“Hybrid : Hybrid models are a common approach for video analysis where a representation learning module is used to distil spatial information in a single image frame and a sequential module is applied to model the temporal correlation across a series of frames [13, 14].”
BackgroundSelf-citationUAV in the advent of the twenties: Where we stand and what is next
Nex, F., Armenakis, C., Cramer, M., Cucci, D. A., Gerke, M., Honkavaara, E., Kukko, A., Persello, C., & Škaloud, J. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.12.006 · Cited by 348
DeepTAM: Deep Tracking and Mapping
Zhou, H., Ummenhofer, B., & Brox, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_50 · Cited by 226
End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks
Wang, S., Clark, R., Wen, H., & Trigoni, N. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917734298 · Cited by 244
“This paper extends the work presented in (Wang et al. 2017) with significantly more details on the architecture and training of the DNN, new contribution on uncertainty estimation, and extensively more evaluations on mobile robots and extended to flying robots and human motion.”
BackgroundSelf-citationLO-Net: Deep Real-Time Lidar Odometry
Li, Q., Chen, S., Wang, C., Li, X. S., Wen, C., Cheng, M., & Li, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00867 · Cited by 234
“Deep learning has achieved promising results on the issues of visual odometry (VO) [36, 46, 43, 39], image-based pose estimation or localization [18, 17, 4], and point cloud classification and segmentation [27, 21].”
BackgroundDeep Learning for Spacecraft Pose Estimation from Photorealistic Rendering
Proenca, P. F., & Gao, Y. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197244 · Cited by 229
“DL has also been successfully applied to visual odometry [24, 25].”
BackgroundMethodInfluentialAn Overview on Visual SLAM: From Tradition to Semantic
Chen, W.-F., Shang, G., Ji, A.-H., Zhou, C., Wang, X., Xu, C., Li, Z., & Hu, K. (2022) · Remote Sensing · doi:10.3390/rs14133010 · Cited by 217
DeepIM: Deep Iterative Matching for 6D Pose Estimation
Li, Y., Wang, G., Ji, X.-Y., Xiang, Y., & Fox, D. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-019-01250-9 · Cited by 821
“We tried using the VGG16 image classification network (Simonyan and Zisserman, 2014) as the backbone network, but the results were very poor, confirming the intuition that a representation related to optical flow is very useful for pose matching (Wang et al., 2017).”
MethodRecurrent Neural Network for (Un-)Supervised Learning of Monocular Video Visual Odometry and Depth
Wang, R., Pizer, S. M., & Frahm, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00570 · Cited by 203
“r(deg/m) t err(%) r err(deg/m) ORB-SLAM [25] 15.30 0.003 3.68 0.005 GeoNet [38] 43.76 0.160 35.60 0.138 Zhou et al. [42] 17.84 0.068 37.91 0.178 Zhan et al. [39] 11.92 0.036 12.62 0.034 DeepVO et al. [34] - - 8.11 0.088 Our unsupervised 9.88 0.034 12.24 0.052 Our supervised 9.30 0.035 7.21 0.039 Table 4: Quantitative…”
InfluentialVisual Odometry Revisited: What Should Be Learnt?
Zhan, H., Weerasekera, C. S., Bian, J.-W., & Reid, I. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197374 · Cited by 201
“[22] propose a recurrent network for learning VO from videos.”
BackgroundYao, S., Zhao, Y., Zhang, A., Su, L., & Abdelzaher, T. (2017) · Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems · doi:10.1145/3131672.3131675 · Cited by 188
“ese models have shown signicant improvements on audio sensing [31], tracking and localization [8, 38, 52, 56], human activity recognition [37, 56], and user identication [31, 56].”
BackgroundDeep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
Deng, Y., Zhang, T., Lou, G., Zheng, X., Jin, J., & Han, Q. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3071405 · Cited by 203
GCNv2: Efficient Correspondence Prediction for Real-Time SLAM
Tang, J., Ericson, L., Folkesson, J. B., & Jensfelt, P. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2927954 · Cited by 188
“In DVSO [19], depth is predicted in a similar way to [2], using a virtual stereo view.”
MethodInfluentialSelective Sensor Fusion for Neural Visual-Inertial Odometry
Chen, C., Rosa, S., Miao, Y., Lu, C. X., Wu, W., Markham, A., & Trigoni, N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01079 · Cited by 178
Self-citationRobust Consistent Video Depth Estimation
Kopf, J., Rong, X., & Huang, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00166 · Cited by 199
“Recent years, numerous learning-based approaches have been proposed to tackle these challenges either via supervised [58, 62, 72, 27] or self-supervised learning [73, 33, 71, 19, 54, 65, 74, 66].”
BackgroundDXSLAM: A Robust and Efficient Visual SLAM System with Deep Features
Li, D., Shi, X., Long, Q., Liu, S., Yang, W., Wang, F.-S., Wei, Q., & Qiao, F. (2020) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros45743.2020.9340907 · Cited by 161
“It is worth noting that there are CNNs for end-to-end pose estimation [25] [26] [27] [28], making feature extraction an implicit task hidden in the networks.”
BackgroundDeep Model-Based 6D Pose Refinement in RGB
Manhardt, F., Kehl, W., Navab, N., & Tombari, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_49 · Cited by 158
Critical Design and Control Issues of Indoor Autonomous Mobile Robots: A Review
Niloy, M. A., Shama, A., Chakrabortty, R. K., Ryan, M. J., Badal, F. R., Tasneem, Z., Ahamed, M. H., Moyeen, S. I., Das, S. K., Ali, M. F., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3062557 · Cited by 172
“DeepVO [290] was able to achieve end-to-end learning with a combination of a CNN and an RNN to extract features from video frames and map them to robot motion.”
MethodAlmalıoğlu, Y., Saputra, M. R. U., de Gusmao, P. P. B., Markham, A., & Trigoni, N. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793512 · Cited by 166
“Supervised deep learning based VO and depth recovery techniques have showed good performance in challenging environments and succesfuly alleviated issues such as scale drift, need for feature extraction and parameter finetuning [5]–[8].”
BackgroundMethodInfluentialSelf-citationPerception and Navigation in Autonomous Systems in the Era of Learning: A Survey
Tang, Y. S., Zhao, C., Wang, J., Zhang, C., Sun, Q., Zheng, W. X., Du, W., Qian, F., & Kurths, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3167688 · Cited by 170
“Considering the dynamics and relations between adjacent pose transformations, Wang et al. [136] and Xue et al. [137] used recurrent neural networks (RNNs) for camera localization.”
BackgroundSupervising the New with the Old: Learning SFM from SFM
Klodt, M., & Vedaldi, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_43 · Cited by 147
“It has been shown that dense depth maps can be robustly estimated from a single image by unsupervised learning [2, 13], and furthermore, depth and camera poses [14].”
BackgroundMapNet: An Allocentric Spatial Memory for Mapping Environments
Henriques, J. F., & Vedaldi, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00884 · Cited by 153
“We also reimplemented the state-of-the-art DeepVO method [24], which is similar but uses a FlowNet CNN [5].”
BackgroundMethodInfluentialTowards Monocular Vision based Obstacle Avoidance through Deep Reinforcement LearningPreprint
Xie, L., Wang, S., Markham, A., & Trigoni, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.09829 · Cited by 163
“Deep learning nowadays has shown its great performance in robotics and computer vision[24, 2, 3].”
BackgroundSelf-citationAutonomous Drone Racing: A Survey
Hanover, D., Loquercio, A., Bauersfeld, L., Romero, Á., Pěnička, R., Song, Y., Cioffi, G., Kaufmann, E., & Scaramuzza, D. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3400838 · Cited by 173
“Recent works [73]–[75] proposed vision-based odometry algorithms that are learned end-to-end.”
BackgroundLIFT-SLAM: A deep-learning feature-based monocular visual SLAM method
Bruno, H. M. S., & Colombini, E. L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.027 · Cited by 141
“Figure 10 shows a qualitative comparison between our algorithm and DeepVO [3].”
BackgroundMethodResultInfluentialLu, C. X., Saputra, M. R. U., Zhao, P., Almalıoğlu, Y., de Gusmão, P. P. B., Chen, C., Sun, K., Trigoni, N., & Markham, A. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3430776 · Cited by 136
“A multitude of such models have been proposed and are widely used with different sensors, including RGB camera [35, 62], depth camera [59, 73], IMU [11, 12] and lidar [44, 64].”
BackgroundMethodSelf-citationTuran, M., Almalıoğlu, Y., Araújo, H. J., Konukoğlu, E., & Sitti, M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.10.014 · Cited by 135
Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality
Li, J., Yang, B., Danpeng, C., Wang, N., Guofeng, Z., & Bao, H. (2019) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2019.07.002 · Cited by 137
DeepSLAM: A Robust Monocular SLAM System With Unsupervised Deep Learning
Li, R., Wang, S., & Gu, D. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.2982096 · Cited by 135
“[18], [19] trained an RCNN to estimate the camera motion.”
MethodSelf-citationA Comprehensive Review of Vision-Based 3D Reconstruction Methods
Zhou, L., Wu, G., Zuo, Y., Chen, X., & Hu, H. (2024) · Sensors · doi:10.3390/s24072314 · Cited by 153
DeepV2D: Video to Depth with Differentiable Structure from MotionPreprint
Teed, Z., & Deng, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04605 · Cited by 166
“Kendall et. al [20, 19] focused on the problem of camera localization, while other work [49, 42, 44, 48, 41] aim to estimate camera motion between a pair of frames.”
BackgroundCAM-Convs: Camera-Aware Multi-Scale Convolutions for Single-View Depth
Facil, J. M., Ummenhofer, B., Zhou, H., Montesano, L., Brox, T., & Civera, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01210 · Cited by 130
“…[47], from single [6] and multiple views [33], using end-to-end networks [21] and fusing with multi-view geometry [8], completing depth maps [45, 42], and estimating geolocation [43, 19], relative motion [38], visual odometry [40, 41], and simultaneous localization and mapping (SLAM) [35, 2, 46].”
BackgroundDistilling Knowledge From a Deep Pose Regressor Network
Saputra, M. R. U., de Gusmão, P. P. B., Almalıoğlu, Y., Markham, A., & Trigoni, N. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00035 · Cited by 124
“In particular, we employ DNN-based VO methods [33, 34, 37, 38, 26, 1] which replaces the conventional VO pipeline based on multiple-view geometry [15, 27] to DNN-based pipeline which automatically learns useful features for estimating 6 Degree-of-Freedom (DoF) camera poses.”
BackgroundMethodInfluentialSelf-citationAdvances in intelligent and autonomous navigation systems for small UAS
Bijjahalli, S., Sabatini, R., & Gardi, A. (2020) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2020.100617 · Cited by 123
Han, L., Lin, Y., Du, G., & Lian, S. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968467 · Cited by 125
“We follow the test instructions [8] designed for EuRoC, which reports the results on MH04 and MH05 as shown in Table II.”
BackgroundMethodVisual SLAM: What Are the Current Trends and What to Expect?
Tourani, A., Bavle, H., Sánchez-López, J. L., & Voos, H. (2022) · Sensors · doi:10.3390/s22239297 · Cited by 133
“DeepVO 20 , an end-to-end VO framework using a Deep Recurrent Convolutional Neural Network (RCNN) architecture for monocular settings, was proposed by Wang et al. [102].”
MethodSelfVIO: Self-supervised deep monocular Visual–Inertial Odometry and depth estimation
Almalıoğlu, Y., Turan, M., Saputra, M. R. U., de Gusmão, P. P. B., Markham, A., & Trigoni, N. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.03.005 · Cited by 127
Self-citationSimultaneous Localization and Mapping (SLAM) for Autonomous Driving: Concept and Analysis
Zheng, S., Jinling Wang, A., Rizos, C., Ding, W., & El‐Mowafy, A. (2023) · Remote Sensing · doi:10.3390/rs15041156 · Cited by 141
“[125] introduced an end-to-end VO algorithm with deep Recurrent Convolutional Neural Networks (RCNNs) by combining CNNs with the RNNs.”
BackgroundMethodInfluentialWang, K., Ma, S., Chen, J., Ren, F., & Lu, J. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3038898 · Cited by 114
“However, now we only use some simple and general techniques, such as CNN, RNN, GAN, and its hybrids [24], [25], [38], [39], [85], [86].”
MethodAcharya, D., Khoshelham, K., & Winter, S. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.02.020 · Cited by 114
AbolDeepIO: A Novel Deep Inertial Odometry Network for Autonomous Vehicles
Esfahani, M. A., Wang, H., Wu, K., & Yuan, S. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2909064 · Cited by 114
A Survey on 3D LiDAR Localization for Autonomous Vehicles
Elhousni, M., & Huang, X.-M. (2020) · 2020 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/iv47402.2020.9304812 · Cited by 107
“2D camera images were first used to try and predict the odometry between a pair of images [7], [23], [52], [54] with results that were more or less acceptable, but still not outperforming the state of the art.”
BackgroundDeepSFM: Structure from Motion via Deep Bundle Adjustment
Wei, X., Zhang, Y., Li, Z., Fu, Y., & Xue, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_14 · Cited by 112
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12567 · Cited by 145
“…the KITTI dataset is a common choice to evaluate visual odometry, but previous works split training and testing data in different ways (e.g. [24], [48], [50] used Sequence 00, 02, 08, 09 as training set, and Sequence 03, 04, 05, 06, 07, 10 as testing set, while [25], [30] used Sequence 00 -…”
BackgroundMethodInfluentialSelf-citationTartanVO: A Generalizable Learning-based VOPreprint
Wang, W., Hu, Y., & Scherer, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.00359 · Cited by 113
“[43] is a supervised method trained on Seq 00, 02, 08, 09.”
MethodVisual SLAM Integration With Semantic Segmentation and Deep Learning: A Review
Pu, H., Luo, J., Wang, G., Huang, T., Liu, H., & Luo, J. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3306371 · Cited by 122
“DeepVO [45] and TartanVO [46] proposed new methods that combine deep learning and visual SLAM.”
MethodVisual SLAM for Automated Driving: Exploring the Applications of Deep Learning
Milz, S., Arbeiter, G., Witt, C., Abdallah, B., & Yogamani, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00062 · Cited by 107
“Figure 3: Weakness of traditional monocular Visual SLAM taken from [64].”
BackgroundBeyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry
Xue, F., Wang, X., Li, S., Wang, Q., Wang, J., & Zha, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00877 · Cited by 109
“Theoretically, the global pose of each view can be recovered by integrating predicted relative poses as Pt = ∏t i=1 Pi,i−1P0 (P0 denotes the origin pose of the world coordinate) just as DeepVO [31] and ESP-VO [32].”
BackgroundMethodInfluentialDeep learning-based spacecraft relative navigation methods: A survey
Song, J., Rondao, D., & Aouf, N. (2022) · Acta Astronautica · doi:10.1016/j.actaastro.2021.10.025 · Cited by 106
“[11], the loss minimises the pose Mean Square Error (MSE), but the attitude is represented via a direction cosine matrix.”
BackgroundMethodState of the Art in Vision-Based Localization Techniques for Autonomous Navigation Systems
Alkendi, Y., Seneviratne, L. D., & Zweiri, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3082778 · Cited by 114
A New Wave in Robotics: Survey on Recent MmWave Radar Applications in Robotics
Harlow, K., Jang, H., Barfoot, T. D., Kim, A., & Heckman, C. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3463504 · Cited by 120
“Lu et al. [73] fused together motion features from inertial methods using IONET [74] and potentially other deep odometry methods, such as DeepVO [75] together with an mmWave subnetwork that uses a CNN feature extractor designed to track useful mmWave features in a manner similar to optical flow.”
MethodDeep Learning for Visual Localization and Mapping: A Survey
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3309809 · Cited by 97
“For example, VO can be achieved by building an end-to-end DNN model to directly approximate the function from images to pose [15], [23], [24], [25], [26].”
Self-citationDeep Neural Network Based Inertial Odometry Using Low-Cost Inertial Measurement Units
Chen, C., Lu, C. X., Wahlström, J., Markham, A., & Trigoni, N. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2960780 · Cited by 98
“When applied in tracking and localization, recent deep approaches [8], [9] demonstrate that deep neural networks (DNNs) are capable of extracting high-level motion representations from raw data, while providing the state-of-art results compared with traditional model based techniques in terms of accuracy and robustnes…”
BackgroundSelf-citationMonocular 3D Object Detection: An Extrinsic Parameter Free Approach
Zhou, Y., He, Y., Zhu, H., Wang, C., Li, H., & Jiang, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00747 · Cited by 98
“With the success of deep neural networks, end-to-end learning-based methods [47, 48, 52, 53] have been proposed to tackle the visual odometry problem.”
BackgroundMultivariate Uncertainty in Deep Learning
Russell, R. L., & Reale, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3086757 · Cited by 92
“[28], using a pre-trained FlowNet [29] convolutional neural network to extract features from input image pairs.”
MethodBeyond Photometric Loss for Self-Supervised Ego-Motion Estimation
Shen, T., Luo, Z., Zhou, L., Deng, H., Zhang, R., Fang, T., & Quan, L. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793479 · Cited by 91
“For real-world applications, pose and depth estimation using CNNs have also been integrated into visual odometry systems [44], [24].”
MethodSurvey on Localization Systems and Algorithms for Unmanned Systems
Yuan, S., Wang, H., & Xie, L. (2020) · Unmanned Systems · doi:10.1142/s230138502150014x · Cited by 91
PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization
Wang, G., Wu, X., Liu, Z., & Wang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01565 · Cited by 92
“To our knowledge, most learning-based methods are on the 2D visual odometry [26, 33, 15, 29, 23, 25] or utilize 2D projection of LiDAR [16, 22, 27, 10, 11] and the LiDAR odometry in 3D point clouds is underexplored.”
Aitken, J. M., Evans, M. H., Worley, R., Edwards, S., Zhang, R., Dodd, T. J., Mihaylova, L. S., & Anderson, S. (2021) · IEEE Access · doi:10.1109/access.2021.3115981 · Cited by 103
Unsupervised Geometry-Aware Deep LiDAR Odometry
Cho, Y., Kim, G., & Kim, A. (2020) · IEEE International Conference on Robotics and Automation · doi:10.1109/icra40945.2020.9197366 · Cited by 75
“Therefore, recent studies have examined learning-based methods but mostly for visual odometry in both a supervised [10, 11] and an unsupervised [12, 13] manner.”
BackgroundDeepTIO: A Deep Thermal-Inertial Odometry With Visual Hallucination
Saputra, M. R. U., de Gusmão, P. P. B., Lu, C. X., Almalıoğlu, Y., Rosa, S., Chen, C., Wahlström, J., Wang, W., Markham, A., & Trigoni, N. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2969170 · Cited by 86
“We use an LSTM to model the long-term temporal dependencies of camera ego-motion as seen in [4], [5].”
MethodSelf-citationEdge Robotics: Edge-Computing-Accelerated Multirobot Simultaneous Localization and Mapping
Huang, P., Zeng, L., Chen, X., Huang, L., Zhou, Z., & Yu, S. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3146461 · Cited by 88
LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation
Costante, G., & Ciarfuglia, T. A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2803211 · Cited by 84
“work. Through this sensor fusion, the resulting algorithm is able to give good results but requires sensors other than a single monocular camera. The use of LSTM is further explored by Wang et al. in [21], this time without any sensor fusion. The resulting architecture gives again good performances on KITTI sequences…”
BackgroundXie, L., Wang, S., Rosa, S., Markham, A., & Trigoni, N. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461203 · Cited by 83
Self-citationVR-Caps: A Virtual Environment for Capsule Endoscopy
İncetan, K., Celik, I. O., Obeid, A., Gokceler, G. I., Ozyoruk, K. B., Almalıoğlu, Y., Chen, R. J., Mahmood, F., Gilbert, H. B., Durr, N. J., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.101990 · Cited by 80
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
Zou, Y., Ji, P., Tran, Q.-H., Huang, J., & Chandraker, M. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_42 · Cited by 83
Shi, J., Zhu, Z., Zhang, J., Liu, R., Wang, Z., Chen, S., & Liu, H. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341147 · Cited by 80
“Next, the obtained global feature pair of each frame is used as an input to the LSTM layer, extracting temporal information between consecutive frames for sequential learning [3].”
MethodDeep Learning Techniques for Visual SLAM: A Survey
Mokssit, S., Licea, D. B., Guermah, B., & Ghogho, M. (2023) · IEEE Access · doi:10.1109/access.2023.3249661 · Cited by 81
“DeepVO[82] solves the visual odometry problem by adding a recurrent neural network (RNN) on top of CNN.”
Self-Supervised Deep Visual Odometry With Online Adaptation
Li, S., Wang, X., Cao, Y., Xue, F., Yan, Z., & Zha, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00637 · Cited by 78
“DeepVO [38] uses recurrent neural network (RNN) to leverage sequential correlations to estimate poses recurrently.”
BackgroundDeep Sensor Fusion Between 2D Laser Scanner and IMU for Mobile Robot Localization
Li, C., Wang, S., Zhuang, Y., & Yan, F. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2910826 · Cited by 77
“2) Deep Learning based methods: Most of existing deep learning based robot localization methods are vision based [17], [18], [19].”
Self-citationLi, G., Yu, L., & Fei, S. (2020) · Measurement · doi:10.1016/j.measurement.2020.108403 · Cited by 80
Pose Graph optimization for Unsupervised Monocular Visual Odometry
Li, Y., Ushiku, Y., & Harada, T. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793706 · Cited by 76
“[11] train a deep recurrent network end-to-end to predict ego-motion using ground truth trajectory as supervision.”
BackgroundMethodUnsuperPoint: End-to-end Unsupervised Interest Point Detector and DescriptorPreprint
Christiansen, P., Kragh, M. F., Brodskiy, Y., & Karstoft, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04011 · Cited by 90
DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction
Chen, C., Lu, C. X., Wang, B., Trigoni, N., & Markham, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3112460 · Cited by 78
Self-citationMasking by Moving: Learning Distraction-Free Radar Odometry from Pose InformationPreprint
Barnes, D., Weston, R., & Posner, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03752 · Cited by 75
“State-of-the-art visual odometry, for example, leverages learnt feature representations [5] as well as attention masks filtering out potentially distracting objects [6].”
BackgroundOngoing Evolution of Visual SLAM from Geometry to Deep Learning: Challenges and Opportunities
Li, R., Wang, S., & Gu, D. (2018) · Cognitive Computation · doi:10.1007/s12559-018-9591-8 · Cited by 80
“Melekhov et al.[73] also presented a relative camera pose estimation system with CNN. Turan et al. proposed Deep EndoVO [74] which is similar to DeepVO [15], and applied it to the area of soft robotics [75].”
InfluentialSelf-citationAerial orthoimage generation for UAV remote sensing: Review
Zhang, J., Xu, S., Zhao, Y., Sun, J., Xu, S., & Zhang, X. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.08.007 · Cited by 76
From SLAM to Situational Awareness: Challenges and Survey
Bavle, H., Sánchez-López, J. L., Cimarelli, C., Tourani, A., & Voos, H. (2023) · Sensors · doi:10.3390/s23104849 · Cited by 88
RETRACTED ARTICLE: A review on quantum computing and deep learning algorithms and their applications
Valdez, F., & Melín, P. (2022) · Soft Computing · doi:10.1007/s00500-022-07037-4 · Cited by 82
RetractedPseudo RGB-D for Self-improving Monocular SLAM and Depth Prediction
Tiwari, L., Ji, P., Tran, Q.-H., Zhuang, B., Anand, S., & Chandraker, M. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_26 · Cited by 70
Deep Learning and Artificial Neural Networks for Spacecraft Dynamics, Navigation and Control
Silvestrini, S., & Lavagna, M. R. (2022) · Drones · doi:10.3390/drones6100270 · Cited by 78
Panoramic SLAM from a multiple fisheye camera rig
Ji, S., Qin, Z., Shan, J., & Lü, M. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.11.014 · Cited by 70
RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild
Zhang, J. Y., Ramanan, D., & Tulsiani, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19821-2_34 · Cited by 77
Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry
Li, S., Xue, F., Wang, X., Yan, Z., & Zha, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00294 · Cited by 70
“Since deep learning captures structural perception by extracting high-level features, a number of learningbased VO methods have been applied to break through the limitations of classic approaches [19, 33, 34, 36, 37, 41].”
BackgroundMethodDeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Network
Wang, W., Saputra, M. R. U., Zhao, P., de Gusmão, P. P. B., Yang, B., Chen, C., Markham, A., & Trigoni, N. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967756 · Cited by 67
Self-citation
Metadata
- Authors
- Wang, S., Clark, R., Wen, H., & Trigoni, N.
- Affiliations
- University of Oxford (GB)
- Venue
- 2017 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2017-05
- Pages
- 2043-2050
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2598706937
- Cited by
- 970
- References
- 29
- Citations by year
- 2026: 63 · 2025: 120 · 2024: 104 · 2023: 123 · 2022: 130 · 2021: 131 · 2020: 125 · 2019: 116 · 2018: 52 · 2017: 6
- FWCI
- 143.15
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M019918/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- monocular visual odometry, end-to-end learning, recurrent convolutional neural network, pose estimation, feature representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
