The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Keyframe-based visual–inertial odometry using nonlinear optimization
Abstract
Combining visual and inertial measurements has become popular in mobile robotics, since the two sensing modalities offer complementary characteristics that make them the ideal choice for accurate visual–inertial odometry or simultaneous localization and mapping (SLAM). While historically the problem has been addressed with filtering, advancements in visual estimation suggest that nonlinear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms. The problem is kept tractable and thus ensuring real-time operation by limiting the optimization to a bounded window of keyframes through marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms. We present evaluation results on complementary datasets recorded with our custom-built stereo visual–inertial hardware that accurately synchronizes accelerometer and gyroscope measurements with imagery. A comparison of both a stereo and monocular version of our algorithm with and without online extrinsics estimation is shown with respect to ground truth. Furthermore, we compare the performance to an implementation of a state-of-the-art stochastic cloning sliding-window filter. This competitive reference implementation performs tightly coupled filtering-based visual–inertial odometry. While our approach declaredly demands more computation, we show its superior performance in terms of accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901 · cited 2×
“Our processing pipeline employs a customized multiscale SSE-optimized Harris corner detector (Harris and Stephens, 1988) followed by BRISK descriptor extraction (Leutenegger et al.”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
BRISK: Binary Robust invariant scalable keypoints
Leutenegger, S., Chli, M., & Siegwart, R. Y. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126542 · Cited by 3,450
Self-citationA Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation
Mourikis, A. I., & Roumeliotis, S. I. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.364024 · Cited by 1,951
Real-time simultaneous localisation and mapping with a single camera
Davison (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238654 · Cited by 1,733
iSAM2: Incremental smoothing and mapping using the Bayes tree
Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J. J., & Dellaert, F. (2012) · The International Journal of Robotics Research · doi:10.1177/0278364911430419 · Cited by 1,362
Unified temporal and spatial calibration for multi-sensor systems
Furgale, P., Rehder, J., & Siegwart, R. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696514 · Cited by 807
Self-citationThe Graph SLAM Algorithm with Applications to Large-Scale Mapping of Urban Structures
Thrun, S., & Montemerlo, M. (2006) · The International Journal of Robotics Research · doi:10.1177/0278364906065387 · Cited by 689
A robust and modular multi-sensor fusion approach applied to MAV navigation
Lynen, S., Achtelik, M. W., Weiss, S., Chli, M., & Siegwart, R. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696917 · Cited by 572
Self-citationVisual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration
Kelly, J., & Sukhatme, G. S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364910382802 · Cited by 536
Real-time monocular SLAM: Why filter?
Strasdat, H., Montiel, J. M. M., & Davison, A. J. (2010) · 2010 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2010.5509636 · Cited by 475
Keyframe-Based Visual-Inertial SLAM using Nonlinear Optimization
Leutenegger, S., Furgale, P., Rabaud, V., Chli, M., Konolige, K., & Siegwart, R. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.037 · Cited by 447
Self-citationMirzaei, F. M., & Roumeliotis, S. I. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2004486 · Cited by 442
Visual-inertial navigation, mapping and localization: A scalable real-time causal approach
Jones, E. S., & Soatto, S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364910388963 · Cited by 435
Weiss, S., Achtelik, M. W., Lynen, S., Chli, M., & Siegwart, R. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6225147 · Cited by 417
Self-citationStrapdown Inertial Navigation Integration Algorithm Design Part 2: Velocity and Position Algorithms
Savage, P. G. (1998) · Journal of Guidance, Control, and Dynamics · doi:10.2514/2.4242 · Cited by 383
Large-Scale Visual Odometry for Rough Terrain
Konolige, K., Agrawal, M., & Solà, J. (2010) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-642-14743-2_18 · Cited by 363
Vision-Aided Inertial Navigation for Spacecraft Entry, Descent, and Landing
Mourikis, A. I., Trawny, N., Roumeliotis, S. I., Johnson, A. E., Ansar, A., & Matthies, L. (2009) · IEEE Transactions on Robotics · doi:10.1109/tro.2009.2012342 · Cited by 345 · cited 3×
“We therefore compare our results with a competitive implementation of this sliding window filter using on-the-fly feature marginalization as published by Mourikis et al. (2009).”
ResultInfluentialA synchronized visual-inertial sensor system with FPGA pre-processing for accurate real-time SLAM
Nikolic, J., Rehder, J., Burri, M., Gohl, P., Leutenegger, S., Furgale, P. T., & Siegwart, R. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6906892 · Cited by 270
Self-citationSliding window filter with application to planetary landing
Sibley, G., Matthies, L., & Sukhatme, G. (2010) · Journal of Field Robotics · doi:10.1002/rob.20360 · Cited by 259
Augmenting inertial navigation with image-based motion estimation
Roumeliotis, S. I., Johnson, A. E., & Montgomery, J. F. (2003) · Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292) · doi:10.1109/robot.2002.1014441 · Cited by 202
RSLAM: A System for Large-Scale Mapping in Constant-Time Using Stereo
Mei, C., Sibley, G., Cummins, M., Newman, P., & Reid, I. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0361-7 · Cited by 193
Improving the accuracy of EKF-based visual-inertial odometry
Li, M., & Mourikis, A. I. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6225229 · Cited by 191
OpenGV: A unified and generalized approach to real-time calibrated geometric vision
Kneip, L., & Furgale, P. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906582 · Cited by 183
Self-citationKeyframe-based Visual-Inertial SLAM using Nonlinear Optimization
Leutenegger, S., Furgale, P. T., Rabaud, V., Chli, M., Konolige, K., & Siegwart, R. (2013) · doi:10.3929/ethz-b-000236658 · Cited by 169
Self-citationReal-time implementation of airborne inertial-SLAM
Kim, J., & Sukkarieh, S. (2006) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2006.06.006 · Cited by 161
Motion Estimation from Image and Inertial Measurements
Strelow, D., & Singh, S. (2004) · The International Journal of Robotics Research · doi:10.1177/0278364904045593 · Cited by 158
Real-time motion tracking on a cellphone using inertial sensing and a rolling-shutter camera
Li, M., Kim, B. H., & Mourikis, A. I. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6631248 · Cited by 157
Factor graph based incremental smoothing in inertial navigation systems
Indelman, V., Williams, S. R., Kaess, M., & Dellaert, F. (2012) · SMARTech Repository (Georgia Institute of Technology) · Cited by 135
A First-Estimates Jacobian EKF for Improving SLAM Consistency
Huang, G. P., Mourikis, A. I., & Roumeliotis, S. I. (2009) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-00196-3_43 · Cited by 126
Motion tracking with fixed-lag smoothing: Algorithm and consistency analysis
Dong-Si, T.-C., & Mourikis, A. I. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5980267 · Cited by 123
Vision based navigation for micro helicopters
Weiß, S. (2012) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-007344020 · Cited by 118
Towards consistent visual-inertial navigation
Huang, G. P., Kaess, M., & Leonard, J. J. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907581 · Cited by 104
Pose estimation using linearized rotations and quaternion algebra
Barfoot, T. D., Forbes, J. R., & Furgale, P. (2010) · Acta Astronautica · doi:10.1016/j.actaastro.2010.06.049 · Cited by 90
Self-citationEstimator initialization in vision-aided inertial navigation with unknown camera-IMU calibration
Dong-Si, T.-C., & Mourikis, A. I. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6386235 · Cited by 85
On the Consistency of Vision-Aided Inertial Navigation
Kottas, D. G., Hesch, J. A., Bowman, S. L., & Roumeliotis, S. I. (2013) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-00065-7_22 · Cited by 76 · cited 2×
“(2012a) and others (Huang et al., 2011; Kottas et al., 2012; Hesch et al., 2012b, 2013; Huang et al., 2013) derived formulations allowing to choose the linearization points of the VINS system in a way such that the observability properties of the linearized and non-linear system are equal.”
BackgroundMartinelli, A. (2011) · IEEE Transactions on Robotics · doi:10.1109/tro.2011.2109210 · Cited by 74
Chai, L., Hoff, W., & Vincent, T. L. (2002) · PRESENCE Virtual and Augmented Reality · doi:10.1162/105474602320935829 · Cited by 73
An observability-constrained sliding window filter for SLAM
Huang, G. P., Mourikis, A. I., & Roumeliotis, S. I. (2011) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2011.6095161 · Cited by 72
Towards Consistent Vision-Aided Inertial Navigation
Hesch, J. A., Kottas, D. G., Bowman, S. L., & Roumeliotis, S. I. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-36279-8_34 · Cited by 71
Factor graph based incremental smoothing in inertial navigation systems
Indelman, Williams, Kaess, & Dellaert (2012) · Cited by 66
C-KLAM: Constrained keyframe-based localization and mapping
Nerurkar, E. D., Wu, K. J., & Roumeliotis, S. I. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907385 · Cited by 65
Airborne smoothing and mapping using vision and inertial sensors
Bryson, M., Johnson‐Roberson, M., & Sukkarieh, S. (2009) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2009.5152678 · Cited by 63
Jia, C., & Evans, B. L. (2012) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2012.6343441 · Cited by 53
Camera trajectory estimation using inertial sensor measurements and structure from motion results
Jung, S., & Taylor, C. J. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2001.991037 · Cited by 49
Vision-aided inertial navigation for resource-constrained systems
Li, M., & Mourikis, A. I. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6386223 · Cited by 41
Fast 3D pose estimation with out-of-sequence measurements
Ranganathan, A., Kaess, M., & Dellaert, F. (2007) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2007.4399318 · Cited by 36
An estimation algorithm for vision-based exploration of small bodies in space
Bayard, D. S., & Brugarolas, P. B. (2005) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2005.1470719 · Cited by 33
Tightly-Coupled Image-Aided Inertial Navigation Using the Unscented Kalman Filter
Ebcin, S., & Veth, M. (2007) · Defense Technical Information Center (DTIC) · Cited by 24
Visual-inertial structure from motion: Observability vs minimum number of sensors
Martinelli, A. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6906979 · Cited by 9
Cited by
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
Qin, T., Li, P., & Shen, S. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2853729 · Cited by 4,669
“To achieve constant processing time, graph-based VIO methods [15], [17], [18] usually optimize over a bounded-size sliding window of recent states by marginalizing out past states and measurements.”
BackgroundMethodInfluentialORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM
Campos, C., Elvira, R., Rodriguez, J. J. G., M. Montiel, J. M., & D. Tardos, J. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3075644 · Cited by 4,274
Engel, J., Koltun, V., & Cremers, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,924
The EuRoC micro aerial vehicle datasets
Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M. W., & Siegwart, R. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915620033 · Cited by 2,066
Self-citationOn-Manifold Preintegration for Real-Time Visual--Inertial Odometry
Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2597321 · Cited by 1,502
Metadata
- Authors
- Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., & Furgale, P.
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2014-12-15
- Volume
- 34
- Issue
- 3
- Pages
- 314-334
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W2091790851
- Cited by
- 1,723
- References
- 51
- FWCI
- 85.69
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.