Towards visual ego-motion learning in robots
Open accessAbstract
Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The online error correction modules increase VIO robustness to spatial misalignments between camera and IMU that affect state-of-the-art traditional VIO methods.
“Additionally, we show how the OEC modules provide increased VIO robustness against spatial misalignments between a camera and IMU that plague SoA traditional approaches to VIO [3], [4], [5].”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893 · see citing works
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
Bundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988 · see citing works
1 year, 1000 km: The Oxford RobotCar dataset
Maddern, W., Pascoe, G., Linegar, C., & Newman, P. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916679498 · Cited by 1,632 · see citing works
Scaramuzza, D., & Fraundorfer, F. (2011) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2011.943233 · Cited by 1,550 · see citing works
iSAM2: Incremental smoothing and mapping using the Bayes tree
Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J. J., & Dellaert, F. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911430419 · Cited by 1,362 · see citing works
Self-citation(1994) · Aston Publications Explorer (Aston University) · Cited by 1,245 · see citing works
Nistér, D., Naroditsky, O., & Bergen, J. R. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315094 · Cited by 1,031 · see citing works
Obstacle avoidance and navigation in the real world by a seeing robot rover
Moravec, H. (2018) · KiltHub Repository · doi:10.1184/r1/6557033 · Cited by 842 · see citing works
Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme
Kitt, B., Geiger, A., & Lategahn, H. (2010) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2010.5548123 · Cited by 409 · see citing works
Real-time stereo visual odometry for autonomous ground vehicles
Howard, A. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4651147 · Cited by 384 · see citing works
Large-Scale Visual Odometry for Rough Terrain
Konolige, K., Agrawal, M., & Solà, J. (2010) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-14743-2_18 · Cited by 373 · see citing works
Real-time monocular visual odometry for on-road vehicles with 1-point RANSAC
Scaramuzza, D., Fraundorfer, F., & Siegwart, R. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152255 · Cited by 283 · see citing works
Scaramuzza, D. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0441-3 · Cited by 210 · see citing works
Exploring Representation Learning With CNNs for Frame-to-Frame Ego-Motion Estimation
Costante, G., Mancini, M., Valigi, P., & Ciarfuglia, T. A. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2015.2505717 · Cited by 203 · see citing works
Learning Visual Odometry with a Convolutional Network
Konda, K., & Memisevic, R. (2015) · International Conference on Computer Vision Theory and Applications · doi:10.5220/0005299304860490 · Cited by 170 · see citing works
Benefit of large field-of-view cameras for visual odometry
Zhang, Z., Rebecq, H., Förster, C., & Scaramuzza, D. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487210 · Cited by 170 · see citing works
Omnidirectional visual odometry for a planetary rover
Corke, P., Strelow, D., & Singh, S. (2005) · 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566) · doi:10.1109/iros.2004.1390041 · Cited by 162 · see citing works
Motion Estimation for Self-Driving Cars with a Generalized Camera
Lee, G. H., Faundorfer, F., & Pollefeys, M. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.354 · Cited by 155 · see citing works
Robust stereo ego-motion for long distance navigation
Olson, C. F., Matthies, L. H., Schoppers, H., & Maimone, M. W. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.854879 · Cited by 149 · see citing works
(1989) · Cited by 133 · see citing works
Ke, Q., & Kanade, T. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211380 · Cited by 83 · see citing works
Using multi-camera systems in robotics: Efficient solutions to the NPnP problem
Kneip, L., Furgale, P. T., & Siegwart, R. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6631107 · Cited by 78 · see citing works
Omnidirectional 3D reconstruction in augmented Manhattan worlds
Schonbein, M., & Geiger, A. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6942637 · Cited by 63 · see citing works
Visual navigation using planar homographies
Liang, B., & Pears, N. E. (2003) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2002.1013362 · Cited by 62 · see citing works
Evaluation of non-geometric methods for visual odometry
Ciarfuglia, T. A., Costante, G., Valigi, P., & Ricci, E. (2014) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.08.001 · Cited by 50 · see citing works
Learning general optical flow subspaces for egomotion estimation and detection of motion anomalies
Roberts, R. B., Potthast, C., & Dellaert, F. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206538 · Cited by 47 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Ronald Clark, Sen Wang, Hongkai Wen, Andrew Markham, and Niki Trigoni. VINet: Visual-Inertial odometry as a sequence-to-sequence learning problem. AAAI, 2016. 2
Cited by
Sorted by citations
Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery
Shamwell, E. J., Lindgren, K., Leung, S., & Nothwang, W. D. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2909895 · Cited by 86 · see citing works
Shamwell, E. J., Leung, S., & Nothwang, W. D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593573 · Cited by 41 · see citing works
- Authors
- Pillai, S., & Leonard, J. J.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- ORCID
- Leonard, John J.
- Venue
- 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2017-09
- Pages
- 5533-5540
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2620334791
- Cited by
- 42
- References
- 28
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 4 · 2022: 5 · 2021: 6 · 2020: 4 · 2019: 12 · 2018: 7
- FWCI
- 11.01
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- visual odometry, mixture density networks, conditional variational autoencoder, optical flow, scene flow, self-supervised learning, robot learning, wheel odometry, camera optics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
