The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Visual-inertial navigation, mapping and localization: A scalable real-time causal approach
Abstract
We describe a model to estimate motion from monocular visual and inertial measurements. We analyze the model and characterize the conditions under which its state is observable, and its parameters are identifiable. These include the unknown gravity vector, and the unknown transformation between the camera coordinate frame and the inertial unit. We show that it is possible to estimate both state and parameters as part of an on-line procedure, but only provided that the motion sequence is ‘rich enough’, a condition that we characterize explicitly. We then describe an efficient implementation of a filter to estimate the state and parameters of this model, including gravity and camera-to-inertial calibration. It runs in real-time on an embedded platform. We report experiments of continuous operation, without failures, re-initialization, or re-calibration, on paths of length up to 30 km. We also describe an integrated approach to ‘loop-closure’, that is the recognition of previously seen locations and the topological re-adjustment of the traveled path. It represents visual features relative to the global orientation reference provided by the gravity vector estimated by the filter, and relative to the scale provided by their known position within the map; these features are organized into ‘locations’ defined by visibility constraints, represented in a topological graph, where loop-closure can be performed without the need to re-compute past trajectories or perform bundle adjustment. The software infrastructure as well as the embedded platform is described in detail in a previous technical report.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
(1981) · Cited by 11,963
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Isidori, A. (1989) · Springer eBooks · doi:10.1007/978-3-662-02581-9 · Cited by 7,869
Stochastic Processes and Filtering Theory
(1970) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(09)x6022-4 · Cited by 7,302
A Mathematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. S. (2017) · CRC Press eBooks · doi:10.1201/9781315136370 · Cited by 6,694
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · cited 1×
“It is inspired by the text search literature, and is similar in spirit to other large-scale visual recognition systems (Sivic and Zisserman 2003; Nister and Stewenius 2006; Ho and Newman 2007; Chum et al. 2009), but adapted to make use of the map.”
Kailath, T. (1980) · Cited by 6,215
Tsai, R. (1987) · IEEE Journal on Robotics and Automation · doi:10.1109/jra.1987.1087109 · Cited by 5,838
New extension of the Kalman filter to nonlinear systems
Julier, S. J., & Uhlmann, J. K. (1997) · SPIE Proceedings · doi:10.1117/12.280797 · Cited by 5,331
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2015) · doi:10.1109/cvpr31182.2015 · Cited by 3,737
Proceedings. 1985 IEEE International Conference on Robotics and Automation
(1985) · doi:10.1109/robot.1985 · Cited by 3,673
Scalable Recognition with a Vocabulary Tree
Nister, D., & Stewenius, H. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.264 · Cited by 3,579
A Mathmematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. (1994) · Cited by 2,676
IEEE Transactions on Automatic Control
(2023) · IEEE Transactions on Automatic Control · doi:10.1109/tac.9 · Cited by 2,649
Computer vision and pattern recognition
Zheng, N., Loizou, G., Jiang, X., Lan, X., & Li, X. (2007) · International Journal of Computer Mathematics · doi:10.1080/00207160701303912 · Cited by 2,463
International Journal of Computer Vision
(2013) · International Journal of Computer Vision · doi:10.1007/11263.1573-1405 · Cited by 2,340
A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation
Mourikis, A. I., & Roumeliotis, S. I. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.364024 · Cited by 1,951
Zhang, J., Marszałek, M., Lazebnik, S., & Schmid, C. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-9794-4 · Cited by 1,923
2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2001) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2001 CVPR-01 · doi:10.1109/cvpr.2001.990916 · Cited by 1,876
Real-time simultaneous localisation and mapping with a single camera
Davison (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238654 · Cited by 1,733
Nonlinear Bayesian estimation using Gaussian sum approximations
Alspach, D., & Sorenson, H. (1972) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1972.1100034 · Cited by 1,239
An Invitation to 3-D Vision: From Images to Geometric Models
Ma, Y., Soatto, S., Koseck, J., & Sastry, S. (2003) · Cited by 846
Self-citationGuivant, J. E., & Nebot, E. M. (2001) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.938382 · Cited by 772
IEEE International Conference On Robotics And Automation
(2001) · Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164) · doi:10.1109/robot.2001.932732 · Cited by 759
Visual odometry for ground vehicle applications
Nistér, D., Naroditsky, O., & Bergen, J. (2006) · Journal of Field Robotics · doi:10.1002/rob.20103 · Cited by 634
City-Scale Location Recognition
Schindler, G., Brown, M., & Szeliski, R. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383150 · Cited by 626
Proceedings of the 2009 European Control conference
Grunnet, J. D., Bak, T., Bendtsen, J. D., & Ankersen, F. (2009) · Cited by 569
An invitation to 3‐d vision: from images to geometric models
Holmgren, D. E. (2004) · The Photogrammetric Record · doi:10.1111/j.0031-868x.2004.295_2.x · Cited by 558
Recursive 3-D road and relative ego-state recognition
Dickmanns, E. D., & Mysliwetz, B. D. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.121789 · Cited by 553
FrameSLAM: From Bundle Adjustment to Real-Time Visual Mapping
Konolige, K., & Agrawal, M. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2004832 · Cited by 532
A complementary filter for attitude estimation of a fixed-wing UAV
Euston, M., Coote, P., Mahony, R. E., Kim, J., & Hamel, T. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4650766 · Cited by 525
Recursive estimation of motion, structure, and focal length
Azarbayejani, A., & Pentland, A. P. (1995) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.387503 · Cited by 484
Preemptive RANSAC for live structure and motion estimation
Nister (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238341 · Cited by 405
Real Time Localization and 3D Reconstruction
Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., & Sayd, P. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) · doi:10.1109/cvpr.2006.236 · Cited by 396
Image Based Localization in Urban Environments
Zhang, W., & Kosecka, J. (2006) · Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06) · doi:10.1109/3dpvt.2006.80 · Cited by 373
Large-Scale Visual Odometry for Rough Terrain
Konolige, K., Agrawal, M., & Solà, J. (2010) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-642-14743-2_18 · Cited by 363
Nonlinear control systems: an introduction (2nd ed.)
Isidori, A. (1989) · Springer eBooks · Cited by 335
Structure from motion causally integrated over time
Chiuso, A., Favaro, P., Jin, H., & Soatto, S. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993559 · Cited by 324
Self-citationBosse, M., Newman, P., Leonard, J., & Teller, S. (2004) · The International Journal of Robotics Research · doi:10.1177/0278364904049393 · Cited by 305
Multi-resolution real-time stereo on commodity graphics hardware
Yang, R., & Pollefeys, M. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211356 · Cited by 273
(2001) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2001 CVPR-01 · doi:10.1109/cvpr.2001.990448 · Cited by 250
Geometric min-Hashing: Finding a (thick) needle in a haystack
Chum, O., Perd'och, M., & Matas, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206531 · Cited by 230
Williams, B., Klein, G., & Reid, I. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4409115 · Cited by 224
Detecting Loop Closure with Scene Sequences
Ho, K. L., & Newman, P. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0020-1 · Cited by 213
Augmenting inertial navigation with image-based motion estimation
Roumeliotis, S. I., Johnson, A. E., & Montgomery, J. F. (2003) · Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292) · doi:10.1109/robot.2002.1014441 · Cited by 202
Omnidirectional Vision Based Topological Navigation
Goedemé, T., Nuttin, M., Tuytelaars, T., & Van Gool, L. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0025-9 · Cited by 198
Feedback control for robotic manipulator with an uncertain Jacobian matrix
Cheah, C. C., Kawamura, S., & Arimoto, S. (1999) · Journal of Robotic Systems · doi:10.1002/(sici)1097-4563(199902)16:2<81::aid-rob2>3.0.co;2-9 · Cited by 135
Monocular SLAM as a Graph of Coalesced Observations
Eade, E., & Drummond, T. W. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4409098 · Cited by 134
A Constant-Time Efficient Stereo SLAM System
Christopher Mei, A., Sibley, G., Cummins, M., Newman, P. M., & Reid, I. D. (2009) · British Machine Vision Conference (BMVC) · doi:10.5244/c.23.54 · Cited by 111
Global localization and relative positioning based on scale-invariant keypoints
Košecká, J., Li, F., & Yang, X. (2005) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2005.03.008 · Cited by 102
A differential geometric approach to nonlinear filtering: the projection filter
Brigo, D., Hanzon, B., & Gland, F. L. (1998) · IEEE Transactions on Automatic Control · doi:10.1109/9.661075 · Cited by 100
Fast wide baseline matching for visual navigation
Goedemé, T., Tuytelaars, T., & Gool, L. V. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315009 · Cited by 99
A semi-direct approach to structure from motion
Jin, H., Favaro, P., & Soatto, S. (2003) · The Visual Computer · doi:10.1007/s00371-003-0202-6 · Cited by 96
Self-citationAccelerated appearance-only SLAM
Cummins, M., & Newman, P. M. (2008) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2008.4543473 · Cited by 95
A dual-layer estimator architecture for long-term localization
Mourikis, A. I., & Roumeliotis, S. I. (2008) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2008.4563131 · Cited by 67
Actionable information in vision
Soatto, S. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459468 · Cited by 53
Self-citationInformation Processing and Management
長谷川, 正. (1984) · Journal of Information Processing and Management · doi:10.1241/johokanri.27.719 · Cited by 53
Fast Relative Pose Calibration for Visual and Inertial Sensors
Kelly, J. W., & Sukhatme, G. S. (2009) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-00196-3_59 · Cited by 52
Jin, H., Favaro, P., & Soatto, S. (2000) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.854954 · Cited by 52
Self-citationFeatures for recognition: viewpoint invariance for non-planar scenes
Vedaldi, A., & Soatto, S. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.99 · Cited by 44
Self-citationTen-fold Improvement in Visual Odometry Using Landmark Matching
Zhu, Z., Oskiper, T., Samarasekera, S., Kumar, R., & Sawhney, H. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4409062 · Cited by 41
On the set of images modulo viewpoint and contrast changes
Sundaramoorthi, G., Petersen, P., Varadarajan, V. S., & Soatto, S. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206704 · Cited by 41
Self-citationVision-based Global Localization Using a Visual Vocabulary
Wang, J., Cipolla, R., & Zha, H. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570770 · Cited by 37
Gauge invariance in projective 3D reconstruction
McLauchlan, P. F. (2003) · doi:10.1109/mview.1999.781081 · Cited by 28
Cited by
Keyframe-based visual–inertial odometry using nonlinear optimization
Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., & Furgale, P. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914554813 · Cited by 1,723
Scaramuzza, D., & Fraundorfer, F. (2011) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2011.943233 · Cited by 1,550
On-Manifold Preintegration for Real-Time Visual--Inertial Odometry
Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2597321 · Cited by 1,502
“The complexity of the EKF grows quadratically in the number of estimated landmarks, therefore, a small number of landmarks (in the order of 20) are typically tracked to allow real-time operation [10]–[12].”
BackgroundRobust visual inertial odometry using a direct EKF-based approach
Bloesch, M., Omari, S., Hutter, M., & Siegwart, R. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353389 · Cited by 917
Metadata
- Authors
- Jones, E. S., & Soatto, S.
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2011-01-17
- Volume
- 30
- Issue
- 4
- Pages
- 407-430
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W2055904838
- Cited by
- 435
- References
- 72
- FWCI
- 56.70
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.