Vision-Aided Absolute Trajectory Estimation Using an Unsupervised Deep Network with Online Error Correction
Open accessAbstract
Adstract- We present an unsupervised deep neural network approach to the fusion of RGB-D imagery with inertial measurements for absolute trajectory estimation. Our network, dubbed the Visual-Inertial-Odometry Learner (VIOLearner), learns to perform visual-inertial odometry (VIO) without inertial measurement unit (IMU) intrinsic parameters (corresponding to gyroscope and accelerometer bias or white noise) or the extrinsic calibration between an IMU and camera. The network learns to integrate IMU measurements and generate hypothesis trajectories which are then corrected online according to the Jacobians of scaled image projection errors with respect to a spatial grid of pixel coordinates. We evaluate our network against state-of-the-art (SOA) visual-inertial odometry, visual odometry, and visual simultaneous localization and mapping (VSLAM) approaches on the KITTI Odometry dataset [1] and demonstrate competitive odometry performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Few heterogeneous datasets of sizes suitable for deep learning on RGB-D data are available.
“While numerous datasets aimed at vision-aided odometry exist [39], [40], [41], [42], [43], [44], few heterogeneous datasets of sizes suitable for deep learning applications on RGB-D data are available.”
From Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery (2020) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2017.2705103 · Cited by 6,898 · see citing works
Jaderberg, M., Simonyan, K., Zisserman, A., & Kavukcuoglu, K. (2015) · arXiv · doi:10.48550/arxiv.1506.02025 · Cited by 5,137 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation
Mourikis, A. I., & Roumeliotis, S. I. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.364024 · Cited by 1,951 · see citing works
Keyframe-based visual–inertial odometry using nonlinear optimization
Leutenegger, S., Lynen, S., Bosse, M. C., Siegwart, R., & Furgale, P. (2014) · The International Journal of Robotics Research · doi:10.1177/0278364914554813 · Cited by 1,718 · see citing works
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632 · see citing works
Nistér, D., Naroditsky, O., & Bergen, J. R. (2004) · Cited by 1,169 · see citing works
Nistér, D., Naroditsky, O., & Bergen, J. R. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315094 · Cited by 1,031 · see citing works
High-precision, consistent EKF-based visual-inertial odometry
Li, M., & Mourikis, A. I. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913481251 · Cited by 813 · see citing works
UnDeepVO: Monocular Visual Odometry Through Unsupervised Deep Learning
Li, R., Wang, S., Long, Z., & Gu, D. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461251 · Cited by 596 · see citing works
Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback
Bloesch, M., Burri, M., Omari, S., Hutter, M., & Siegwart, R. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917728574 · Cited by 485 · see citing works
VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem
Clark, R., Wang, S., Wen, H., Markham, A., & Trigoni, N. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11215 · Cited by 346 · see citing works
Range perception through apparent image speed in freely flying honeybees
Srinivasan, M. V., Lehrer, M., Kirchner, W. H., & Zhang, S. W. (1991) · Visual Neuroscience · doi:10.1017/s095252380000136x · Cited by 301 · see citing works
Unsupervised Learning of Depth and Ego-Motion from VideoPreprint
Zhou, T., Brown, M., Snavely, N., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07813 · Cited by 217 · see citing works
Hierarchical Model-Based Motion Estimation
Anandan, P., Bergen, J. R., Hanna, K. J., & Hingorani, R. (1993) · Motion Analysis and Image Sequence Processing · doi:10.1007/978-1-4615-3236-1_1 · Cited by 193 · see citing works
VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem
Clark, R., Wang, S., Wen, H., Markham, A., & Trigoni, N. (2017) · Warwick Research Archive Portal (University of Warwick) · doi:10.5555/3298023.3298149 · Cited by 102 · see citing works
UnDeepVO: Monocular Visual Odometry through Unsupervised Deep LearningPreprint
Li, R., Wang, S., Long, Z., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06841 · Cited by 48 · see citing works
Towards visual ego-motion learning in robots
Pillai, S., & Leonard, J. J. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206441 · Cited by 42 · see citing works
Enabling aggressive motion estimation at low-drift and accurate mapping in real-time
Zhang, J., & Singh, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989589 · Cited by 31 · see citing works
Depth enhanced visual-inertial odometry based on Multi-State Constraint Kalman Filter
Pang, F., Chen, Z., Pu, L., & Wang, T. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205989 · Cited by 8 · see citing works
Shamwell, E. J., Nothwang, W. D., & Perlis, D. (2018) · Sensors · doi:10.3390/s18051427 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- M. N. Galfond, "Visual-inertial odometry with depth sensing using a multi-state constraint Kalman filter," Master's thesis, Massachusetts Institute of Technology, 2014.
- E. J. Shamwell, W. D. Nothwang, and D. Perlis, "A deep neural network approach to fusing vision and heteroscedastic motion esti- mates for low-SWaP robotic applications," in Multisensor Fusion and Integration for Intelligent Systems (MFI). IEEE, 2017.
- E. J. Shamwell, W. Nothwang, and D. Perlis, "Multi-hypothesis visual inertial flow," IEEE Robotics and Automation Letters, Submitted.
Cited by
Sorted by citations
Depth From Videos in the Wild: Unsupervised Monocular Depth Learning From Unknown Cameras
Gordon, A., Li, H., Jonschkowski, R., & Angelova, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00907 · Cited by 389 · see citing works
Chen, C., Zhu, H. J., Li, M., & You, S. (2018) · Robotics · doi:10.3390/robotics7030045 · Cited by 131 · see citing works
Han, L., Lin, Y., Du, G., & Lian, S. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968467 · Cited by 125 · see citing works
SelfVIO: Self-supervised deep monocular Visual–Inertial Odometry and depth estimation
Almalıoğlu, Y., Turan, M., Saputra, M. R. U., de Gusmão, P. P. B., Markham, A., & Trigoni, N. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.03.005 · Cited by 121 · see citing works
Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery
Shamwell, E. J., Lindgren, K., Leung, S., & Nothwang, W. D. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2909895 · Cited by 86 · see citing works
Self-citationLiDAR, IMU, and camera fusion for simultaneous localization and mapping: a systematic review
Fan, Z., Zhang, L.-L., Wang, X.-Y., Shen, Y., & Deng, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11187-w · Cited by 80 · see citing works
OdoNet: Untethered Speed Aiding for Vehicle Navigation Without Hardware Wheeled Odometer
Tang, H., Niu, X., Zhang, T., Li, Y., & Liu, J. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3169549 · Cited by 55 · see citing works
Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
Yang, M., Chen, Y., & Kim, H.-S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_14 · Cited by 51 · see citing works
Neural Network-Based Recent Research Developments in SLAM for Autonomous Ground Vehicles: A Review
Saleem, H., Malekian, R., & Munir, H. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3273913 · Cited by 45 · see citing works
Deep Learning for Visual SLAM: The State-of-the-Art and Future Trends
Favorskaya, M. N. (2023) · Electronics · doi:10.3390/electronics12092006 · Cited by 43 · see citing works
Unsupervised Monocular Visual-inertial Odometry Network
Peng, W., Hua, G., Huang, W., Meng, F., & Liu, H. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/325 · Cited by 35 · see citing works
ATVIO: Attention Guided Visual-Inertial Odometry
Liu, L., Li, G., & Li, T. H. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413912 · Cited by 35 · see citing works
Tightly-Coupled Fusion of VINS and Motion Constraint for Autonomous Vehicle
Yu, Z., Zhu, L., & Lu, G. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3162155 · Cited by 31 · see citing works
Snake-SLAM: Efficient Global Visual Inertial SLAM using Decoupled Nonlinear Optimization
Rückert, D., & Stamminger, M. (2021) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas51884.2021.9476760 · Cited by 20 · see citing works
Synthetic Deep Neural Network Design for Lidar-inertial Odometry Based on CNN and LSTM
Son, H., Lee, B., & Sung, S. (2021) · International Journal of Control Automation and Systems · doi:10.1007/s12555-020-0443-2 · Cited by 14 · see citing works
VINS-Motion: Tightly-coupled Fusion of VINS and Motion Constraint
Yu, Z., Zhu, L., & Lu, G. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9562103 · Cited by 13 · see citing works
CMIF-VIO: A Novel Cross Modal Interaction Framework for Visual Inertial Odometry
Wang, Z., Zhang, Y., Xu, X., Xiong, M., Su, X., & Meng, F. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3511374 · Cited by 10 · see citing works
Research on Low-Light Environment SLAM System via Retinex Enhancement and Deep Feature Optimization
Jiang, K., Zhu, C., & Yang, J. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3581532 · Cited by 9 · see citing works
Rapid SLAM Method for Star Surface Rover in Unstructured Space Environments
Zhang, Z., Cheng, Y., Bu, L., & Ye, J. (2024) · Aerospace · doi:10.3390/aerospace11090768 · Cited by 6 · see citing works
Lindgren, K., Leung, S. M., Nothwang, W. D., & Shamwell, E. J. (2019) · International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar46387.2019.8981570 · Cited by 6 · see citing works
Self-citationLearning Multiplicative Interactions with Bayesian Neural Networks for Visual-Inertial OdometryPreprint
Shinde, K., Lee, J., Humt, M., Sezgin, A., & Triebel, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07630 · Cited by 9 · see citing works
RWKV-VIO: An Efficient and Low-Drift Visual–Inertial Odometry Using an End-to-End Deep Network
Yang, J., Xu, X., Xu, Z., Wu, Z., & Chu, W. (2025) · Sensors · doi:10.3390/s25185737 · Cited by 4 · see citing works
Wang, Y., Yang, H., Cai, J., Wang, G., Wang, J., & Huang, Y. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160277 · Cited by 4 · see citing works
Multi-Sensor Fusion Self-Supervised Deep Odometry and Depth Estimation
Wan, Y., Zhao, Q., Guo, C., Xu, C., & Fang, L. (2022) · Remote Sensing · doi:10.3390/rs14051228 · Cited by 5 · see citing works
Pereira, T., Viegas, C. X. P., Soares, S., & Ferreira, N. (2026) · Robotics · doi:10.3390/robotics15020035 · Cited by 6 · see citing works
Automated Playbook for UAV Traffic Management Based on Spatiotemporal Scenario Data
He, C., Wan, Y., & Xie, J. (2021) · Unmanned Systems · doi:10.1142/s2301385022500145 · Cited by 2 · see citing works
Monocular weakly supervised depth and pose estimation method based on multi-information fusion
Zhang, Z., Qiao, J., & Lin, S. (2021) · Kuwait Journal of Science · doi:10.48129/kjs.12929 · Cited by 1 · see citing works
Agrawal, A., Bhise, A. A., Arasanipalai, R., Tony, L. A., Jana, S., & Ghose, D. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-28715-2_4 · Cited by 2 · see citing works
A Survey of Visual SLAM Based on RGB-D Images Using Deep Learning and Comparative Study for VOE
Le, V.-H., & Nguyen, T.-H.-P. (2025) · Algorithms · doi:10.3390/a18070394 · Cited by 1 · see citing works
SS-VIO: A Novel Cross Modal State Space Visual Inertial Odometry
Qin, W., Lin, Z., Bai, L., Yao, S.-J., Xu, H., & Dong, X. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3643327 · Cited by 1 · see citing works
Sef-VIO: Self-attention-based Multimodal Fusion for Neural Visual-Inertial Odometry
Wang, L., Gu, C., Yang, J., & Chen, Y. (2025) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac66630.2025.11150142 · Cited by 1 · see citing works
View Loss Evaluation and Keyframe Reselection for Active Aerial Visual Reconstruction
Li, Y., Wang, H., Zhang, X., Liu, J., & Chen, X. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739567 · Cited by 1 · see citing works
A Lightweight Sensor Fusion for Neural Visual Inertial Odometry
Lu, Y., Yin, X., Qin, F., Huang, K., Zhang, M., & Huang, W. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-5847-4_4 · Cited by 1 · see citing works
Lidar SLAM-Enabled Precision Detection of Coal Bunker Structural Damage
Yan, J. (2024) · Academic Journal of Engineering and Technology Science · doi:10.25236/ajets.2024.070305 · Cited by 1 · see citing works
Robust Vision-Aided Self-Localization of Mobile Robots
Lindgren, K. (2020) · ResearchWorks at the University of Washington (University of Washington)
Dynamic Motion Modeling for Enhanced Visual-Inertial Odometry
Ye, X., Wang, H., Xu, R., & Li, H. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733475
SATVIO: Stereo Attention-Based Visual Inertial Odometry
Doorshi, R., Saleem, H., & Malekian, R. (2025) · IEEE Journal of Selected Areas in Sensors · doi:10.1109/jsas.2025.3601056
SimCLR-enhanced transformer for smooth and robust indoor VLP trajectory estimation
Das, S., & Sen, R. (2026) · Engineering Research Express · doi:10.1088/2631-8695/ae3b08
MixVIO: Selective Visual-Inertial Odometry via Phase-Aware Attention and Hybrid Kernel Modeling
Ma, L., Zhang, L., Huang, C., Zhang, T., & Chu, X. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3707325
Diff-VIO: A Diffusion Model-Based Pose Optimizer for Visual Inertial Odometry
Qin, W., Kong, X., Zhang, S., Xu, H., & Dong, X. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11697508
RetiGlue-SLAM: A Robust Visual SLAM System Designed for Low-Light and Complex Environments
Li, Y., Liang, X., Guo, G., Su, P., & An, Q. (2026) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi15100468
- Authors
- Shamwell, E. J., Leung, S., & Nothwang, W. D.
- Affiliations
- DEVCOM Army Research Laboratory (US)
- ORCID
- Shamwell, E. Jared
- Venue
- 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2018-10
- Pages
- 2524-2531
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2792489285
- Cited by
- 41
- References
- 21
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 3 · 2023: 5 · 2022: 5 · 2021: 7 · 2020: 3 · 2019: 4 · 2018: 1
- FWCI
- 6.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Aerospace Engineering › Inertial Sensor and Navigation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- visual-inertial odometry, unsupervised deep learning, IMU bias estimation, online error correction, visual odometry, visual SLAM
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
