The DOI matches its Crossref record; 62 of the 66 references it cites were found in a registry or an index.
Visual Odometry Revisited: What Should Be Learnt?
Open accessAbstract
In this work we present a monocular visual odometry (VO) algorithm which leverages geometry-based methods and deep learning. Most existing VO/SLAM systems with superior performance are based on geometry and have to be carefully designed for different application scenarios. Moreover, most monocular systems suffer from scale-drift issue. Some recent deep learning works learn VO in an end-to-end manner but the performance of these deep systems is still not comparable to geometry-based methods. In this work, we revisit the basics of VO and explore the right way for integrating deep learning with epipolar geometry and Perspective-n-Point (PnP) method. Specifically, we train two convolutional neural networks (CNNs) for estimating single-view depths and two-view optical flows as intermediate outputs. With the deep predictions, we design a simple but robust frame-to-frame VO algorithm (DF-VO) which outperforms pure deep learning-based and geometry-based methods. More importantly, our system does not suffer from the scale-drift issue being aided by a scale consistent single-view depth CNN. Extensive experiments on KITTI dataset shows the robustness of our system and a detailed ablation study shows the effect of different factors in our system. Code is available at here: DF-VO.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“As shown in (Zhan et al. 2020), DF-VO with this correspondence selection scheme has already outperformed existing VO/SLAM baselines.”
From DF-VO: What Should Be Learnt for Visual Odometry? (2021) · cites this paper as Zhan et al. 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
(2017) · Cited by 11,625
ORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2705103 · Cited by 6,898 · cited 9×
“One of the most successful and accurate full SLAM systems using a sparse (ORB) feature-based approach is ORB-SLAM2 [13], along with DSO [16], a direct keyframe-based sparse SLAM method.”
MethodInfluentialORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.179 · Cited by 3,177
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088
Engel, J. J., Koltun, V., & Cremers, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,995
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00931 · Cited by 2,801
Multiple View Geometry in Computer Vision (2nd ed)
Hartley, R. I., & Zisserman, A. (2003) · Cambridge University Press eBooks · Cited by 2,747
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Least-squares estimation of transformation parameters between two point patterns
Umeyama, S. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.88573 · Cited by 2,429
SVO: Fast semi-direct monocular visual odometry
Forster, C., Pizzoli, M., & Scaramuzza, D. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906584 · Cited by 2,205
Deeper Depth Prediction with Fully Convolutional Residual Networks
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.32 · Cited by 2,105
DTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
Deep Ordinal Regression Network for Monocular Depth Estimation
Fu, H., Gong, M., Wang, C., Batmanghelich, K., & Tao, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00214 · Cited by 1,991
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632
Self-citationScaramuzza, D., & Fraundorfer, F. (2011) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2011.943233 · Cited by 1,550
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
Yin, Z., & Shi, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00212 · Cited by 1,280
Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields
Liu, F., Shen, C., Lin, G., & Reid, I. D. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505283 · Cited by 1,231
Self-citationStereoScan: Dense 3d reconstruction in real-time
Geiger, A., Ziegler, J., & Stiller, C. (2011) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2011.5940405 · Cited by 1,119
The interpretation of structure from motion
Ullman, S. (1979) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.1979.0006 · Cited by 1,059
Deepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks
Wang, S., Clark, R., Wen, H., & Trigoni, N. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989236 · Cited by 970 · cited 1×
“[22] propose a recurrent network for learning VO from videos.”
BackgroundSVO: Semidirect Visual Odometry for Monocular and Multicamera Systems
Förster, C., Zhang, Z., Gassner, M., Werlberger, M., & Scaramuzza, D. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2623335 · Cited by 919
Deep convolutional neural fields for depth estimation from a single image
Liu, F., Shen, C., & Lin, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299152 · Cited by 892
LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation
Hui, T.-W., Tang, X., & Loy, C. C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00936 · Cited by 817
CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction
Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.695 · Cited by 800
DeMoN: Depth and Motion Network for Learning Monocular Stereo
Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.596 · Cited by 712
Zhan, H., Garg, R., Weerasekera, C. S., Li, K., Agarwal, H., & Reid, I. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00043 · Cited by 705
Self-citationRanjan, A., Jampani, V., Balles, L., Kim, K., Sun, D., Wulff, J., & Black, M. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01252 · Cited by 659
In defence of the 8-point algorithm
Hartley, R. I. (2002) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466816 · Cited by 648
DSAC — Differentiable RANSAC for Camera Localization
Brachmann, E., Krull, A., Nowozin, S., Shotton, J., Michel, F., Gumhold, S., & Rother, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.267 · Cited by 647
GMS: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence
Bian, J.-W., Lin, W.-Y., Matsushita, Y., Yeung, S.-K., Nguyen, T.-D., & Cheng, M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.302 · Cited by 621
Self-citationUnDeepVO: Monocular Visual Odometry Through Unsupervised Deep Learning
Li, R., Wang, S., Long, Z., & Gu, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461251 · Cited by 596
UnFlow: Unsupervised Learning of Optical Flow With a Bidirectional Census Loss
Meister, S., Hur, J., & Roth, S. (2018) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v32i1.12276 · Cited by 535
Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular VideoPreprint
Bian, J.-W., Li, Z., Wang, N., Zhan, H., Shen, C., Cheng, M., & Reid, I. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10553 · Cited by 472
Self-citationAgrawal, P., Carreira, J., & Malik, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.13 · Cited by 438
Learning Less is More - 6D Camera Localization via 3D Surface Regression
Brachmann, E., & Rother, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00489 · Cited by 399
Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry
Yang, N., Wang, R., Stückler, J., & Cremers, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_50 · Cited by 346
An efficient solution to the five-point relative pose problem
Nister, D. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211470 · Cited by 279
The Interpretation of Structure From Motion
(1979) · The MIT Press eBooks · doi:10.7551/mitpress/3877.003.0009 · Cited by 278
DeepTAM: Deep Tracking and Mapping
Zhou, H., Ummenhofer, B., & Brox, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_50 · Cited by 226
Matthies, L. H., Maimone, M. W., Johnson, A., Cheng, Y., Willson, R. G., Villalpando, C. Y., Goldberg, S., Huertas, A., Stein, A. N., & Angelova, A. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0046-z · Cited by 193
Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations
Nekrasov, V., Dharmasiri, T., Spek, A., Drummond, T. W., Shen, C., & Reid, I. D. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794220 · Cited by 149
Self-citationGMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence
Bian, J.-W., Lin, W.-Y., Liu, Y., Zhang, L., Yeung, S.-K., Cheng, M., & Reid, I. D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01280-3 · Cited by 129
Self-citationCNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction
Loo, S. Y., Amiri, A. J., Mashohor, S., Tang, S. H., & Zhang, H. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794425 · Cited by 107
Pose Graph optimization for Unsupervised Monocular Visual Odometry
Li, Y., Ushiku, Y., & Harada, T. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793706 · Cited by 76
An Evaluation of Feature Matchers for Fundamental Matrix EstimationPreprint
Bian, J.-W., Wu, Y.-H., Zhao, J. Y., Liu, Y., Zhang, L., Cheng, M., & Reid, I. D. (2019) · arXiv (Cornell University) · doi:10.5244/c.33.89 · Cited by 70
Self-citationSelf-supervised Learning for Single View Depth and Surface Normal Estimation
Zhan, H., Weerasekera, C. S., Garg, R., & Reid, I. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793984 · Cited by 31
Self-citationENG: End-to-End Neural Geometry for Robust Depth and Pose Estimation Using CNNs
Dharmasiri, T., Spek, A., & Drummond, T. W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20887-5_39 · Cited by 16
Research works printed but not linked: 2
- godard, 2019, Digging into self-supervised monocular depth prediction, IEEE International Conference on Computer Vision (ICCV)
- 10.1023/A:1007941100561 · DOI
Cited by
Sorted by citations
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438
“Recently, there have been many efforts to address this by leveraging deep neural networks [48, 68, 80, 83].”
BackgroundResultTowards Better Generalization: Joint Depth-Pose Learning Without PoseNet
Zhao, W., Liu, S., Shu, Y., & Liu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00917 · Cited by 186
“A recent paper [64] employs similar visual odometry inference strategies to utilize neural network predictions.”
MethodLocalization as a Key Enabler of 6G Wireless Systems: A Comprehensive Survey and an Outlook
Trevlakis, S. E., Boulogeorgos, A. A., Pliatsios, D., Querol, J., Ntontin, K., Sarigiannidis, P. G., Chatzinotas, S., & Renzo, M. D. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3324952 · Cited by 114
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12567 · Cited by 145
“Zhan et al. [67] integrate the learned depth and optical flow predictions into a conventional visual odometry model, achieving competitive performance over other baselines.”
BackgroundTartanVO: A Generalizable Learning-based VOPreprint
Wang, W., Hu, Y., & Scherer, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.00359 · Cited by 113
“Recently, Zhan et al. [29] use PnP techniques to explicitly solve for the scale factor.”
Metadata
- Authors
- Zhan, H., Weerasekera, C. S., Bian, J.-W., & Reid, I.
- Affiliations
- The University of Adelaide (AU)
- Venue
- 2020 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2020-05
- Pages
- 4203-4210
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W3091687063
- Cited by
- 201
- References
- 66
- Citations by year
- 2026: 18 · 2025: 28 · 2024: 37 · 2023: 37 · 2022: 33 · 2021: 30 · 2020: 17
- FWCI
- 43.12
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Australian Centre for Robotic Vision
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Earth and Planetary Sciences › Geology › 3D Surveying and Cultural Heritage
- Keywords
- monocular visual odometry, scale drift, single-view depth estimation, optical flow, epipolar geometry, perspective-n-point, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
