The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration
Abstract
Visual and inertial sensors, in combination, are able to provide accurate motion estimates and are well suited for use in many robot navigation tasks. However, correct data fusion, and hence overall performance, depends on careful calibration of the rigid body transform between the sensors. Obtaining this calibration information is typically difficult and time-consuming, and normally requires additional equipment. In this paper we describe an algorithm, based on the unscented Kalman filter, for self-calibration of the transform between a camera and an inertial measurement unit (IMU). Our formulation rests on a differential geometric analysis of the observability of the camera—IMU system; this analysis shows that the sensor-to-sensor transform, the IMU gyroscope and accelerometer biases, the local gravity vector, and the metric scene structure can be recovered from camera and IMU measurements alone. While calibrating the transform we simultaneously localize the IMU and build a map of the surroundings, all without additional hardware or prior knowledge about the environment in which a robot is operating. We present results from simulation studies and from experiments with a monocular camera and a low-cost IMU, which demonstrate accurate estimation of both the calibration parameters and the local scene structure.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
Isidori, A. (1989) · Springer eBooks · doi:10.1007/978-3-662-02581-9 · Cited by 7,869
Isidori, A. (1995) · Communications and Control Engineering · doi:10.1007/978-1-84628-615-5 · Cited by 6,755
Unscented Filtering and Nonlinear Estimation
Julier, S. J., & Uhlmann, J. K. (2004) · Proceedings of the IEEE · doi:10.1109/jproc.2003.823141 · Cited by 6,588
(2012) · SpringerReference · doi:10.1007/springerreference_117570 · Cited by 4,486
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
Closed-form solution of absolute orientation using unit quaternions
Horn, B. K. P. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.000629 · Cited by 4,116
MonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · cited 1×
“For example, Davison et al. (2007) describe an extended Kalman filter (EKF)-based system that is able to localize a camera in a room-sized environment.”
BackgroundStrapdown Inertial Navigation Technology
Titterton, D., & Weston, J. (2004) · Institution of Engineering and Technology eBooks · doi:10.1049/pbra017e · Cited by 3,348
Stochastic models, estimation, and control
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6186-7 · Cited by 3,229
Aircraft Control and Simulation
Stevens, B., & Lewis, F. L. (2004) · Aircraft Engineering and Aerospace Technology · doi:10.1108/aeat.2004.12776eae.001 · Cited by 3,167
Vlfeat: an open and portable library of computer vision algorithms
Vedaldi, A., & Fulkerson, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874249 · Cited by 2,664
Nonlinear controllability and observability
Hermann, R., & Krener, A. (1977) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1977.1101601 · Cited by 2,374 · cited 5×
“if, for each point x0 ∈ M , there exists an open neighborhood U ⊆ M such that for every open neighborhood V of x0, V ⊆ U , there is no other state x1 ∈ V that is indistinguishable from x0 (Hermann and Krener 1977).”
BackgroundMethodInfluentialVidyasagar, M., & Desoer, C. A. (1980) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1980.4308549 · Cited by 2,011
A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation
Mourikis, A. I., & Roumeliotis, S. I. (2007) · Proceedings 2007 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2007.364024 · Cited by 1,951
Sastry, S. (1999) · Interdisciplinary Applied Mathematics · doi:10.1007/978-1-4757-3108-8 · Cited by 1,764
Progress in Astronautics and Aeronautics
(1964) · Progress in Astronautics and Rocketry · doi:10.1016/b978-1-4832-2730-6.50001-4 · Cited by 1,424
Kuipers, J. B. (1999) · Princeton University Press eBooks · doi:10.1515/9780691211701 · Cited by 1,290
The scaled unscented transformation
Julier, S. J. (2002) · Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301) · doi:10.1109/acc.2002.1025369 · Cited by 1,114
The global positioning system and inertial navigation
Farrell, J. A., & Barth, M. (1999) · Virtual Defense Library (Ministerio de Defensa) · Cited by 1,048
Ma, Y., Soatto, S., Košecká, J., & Sastry, S. S. (2004) · Interdisciplinary Applied Mathematics · doi:10.1007/978-0-387-21779-6 · Cited by 893
Unscented Filtering for Spacecraft Attitude Estimation
Crassidis, J. L., & Markley, F. L. (2003) · Journal of Guidance, Control, and Dynamics · doi:10.2514/2.5102 · Cited by 880
An Invitation to 3-D Vision: From Images to Geometric Models
Ma, Y., Soatto, S., Koseck, J., & Sastry, S. (2003) · Cited by 846
Filtering and System Identification: A Least Squares Approach
Verhaegen, M., & Verdult, V. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511618888 · Cited by 820
Inverse Depth Parametrization for Monocular SLAM
Civera, J., Davison, A. J., & Montiel, J. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2003276 · Cited by 779
Aided Navigation: GPS with High Rate Sensors
Farrell, J. A. (2008) · Cited by 776
Strapdown Inertial Navigation Technology, Second Edition
Titterton, D. H., & Weston, J. (2005) · Cited by 572
Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94)
(1994) · Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94) · doi:10.1109/iros.1994.407417 · Cited by 551
McNeff, J. G. (2002) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/22.989949 · Cited by 508
Mirzaei, F. M., & Roumeliotis, S. I. (2008) · IEEE Transactions on Robotics · doi:10.1109/tro.2008.2004486 · Cited by 442
Fundamentals Of High Accuracy Inertial Navigation
Chatfield, A. B. (1997) · American Institute of Aeronautics and Astronautics eBooks · doi:10.2514/4.866463 · Cited by 427
(1999) · Choice Reviews Online · doi:10.5860/choice.37-0370 · Cited by 398
Eade, E., & Drummond, T. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.263 · Cited by 396
Unified Inverse Depth Parametrization for Monocular SLAM
Montiel, J., Civera, J., & Davison, A. (2006) · Robotics: Science and Systems II · doi:10.15607/rss.2006.ii.011 · Cited by 377
Quaternion kinematic and dynamic differential equations
Chou, J. C. K. (1992) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.127239 · Cited by 374
Vision-Aided Inertial Navigation for Spacecraft Entry, Descent, and Landing
Mourikis, A. I., Trawny, N., Roumeliotis, S. I., Johnson, A. E., Ansar, A., & Matthies, L. (2009) · IEEE Transactions on Robotics · doi:10.1109/tro.2009.2012342 · Cited by 345 · cited 1×
“A variety of hybrid visual–inertial systems have been developed, for applications from short-range sensor localization (Strelow and Singh 2002) to long-range motion estimation for spacecraft entry, descent and landing (Mourikis et al. 2009).”
BackgroundStructure from motion causally integrated over time
Chiuso, A., Favaro, P., Jin, H., & Soatto, S. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993559 · Cited by 324
Sigma-Point Kalman Filters for Integrated Navigation
van der Merwe, R., & Wan, E. A. (2004) · Cited by 184
Motion Estimation from Image and Inertial Measurements
Strelow, D., & Singh, S. (2004) · The International Journal of Robotics Research · doi:10.1177/0278364904045593 · Cited by 158
Using saddle points for subpixel feature detection in camera calibration targets
Lucchese, L., & Mitra, S. K. (2003) · IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) · doi:10.1109/apccas.2002.1115151 · Cited by 93
Combined Visual and Inertial Navigation for an Unmanned Aerial Vehicle
Kelly, J. W., Saripalli, S., & Sukhatme, G. S. (2008) · Springer tracts in advanced robotics · doi:10.1007/978-3-540-75404-6_24 · Cited by 83
Self-citationRelative position sensing by fusing monocular vision and inertial rate sensors
Rock, S. M., & Huster, A. (2003) · Cited by 80
Simultaneous Motion and Structure Estimation by Fusion of Inertial and Vision Data
Gemeiner, P., Einramhof, P., & Vincze, M. (2007) · The International Journal of Robotics Research · doi:10.1177/0278364907080058 · Cited by 67
Generalized architecture for simultaneous localization, auto-calibration, and map-building
Foxlin, E. M. (2003) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/irds.2002.1041444 · Cited by 60
Automatic self-calibration of a vision system during robot motion
Martinelli, A., Scaramuzza, D., & Siegwart, R. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2006.1641159 · Cited by 38
Observability Properties and Optimal Trajectories for On-line Odometry Self-Calibration
Martinelli, A., & Siegwart, R. (2006) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2006.377161 · Cited by 32
Proceedings Sixth IEEE Workshop on Applications of Computer Vision (WACV 2002)
(2002) · Proceedings · doi:10.1109/acv.2002.1182134 · Cited by 14
Theoretical Results on On-line Sensor Self-Calibration
Martinelli, A., Weingarten, J. D., & Siegwart, R. (2006) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2006.282015 · Cited by 7
Cited by
Keyframe-based visual–inertial odometry using nonlinear optimization
Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., & Furgale, P. (2015) · The International Journal of Robotics Research · doi:10.1177/0278364914554813 · Cited by 1,723
Robust visual inertial odometry using a direct EKF-based approach
Bloesch, M., Omari, S., Hutter, M., & Siegwart, R. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353389 · Cited by 917
High-precision, consistent EKF-based visual-inertial odometry
Li, M., & Mourikis, A. I. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913481251 · Cited by 813
“…the EKF Jacobians are computed, even though the IMU’s rotation about gravity (the yaw) is not observable in VIO (see, e.g., (Jones and Soatto, 2011; Kelly and Sukhatme, 2011; Martinelli, 2012)), it appears to be observable in the linearized system model used by the MSCKF, and the same occurs in…”
BackgroundMethodInfluentialUnified temporal and spatial calibration for multi-sensor systems
Furgale, P., Rehder, J., & Siegwart, R. (2013) · 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2013.6696514 · Cited by 807
Metadata
- Authors
- Kelly, J., & Sukhatme, G. S.
- Venue
- The International Journal of Robotics Research
- Publisher
- SAGE Publications
- Published
- 2010-11-05
- Volume
- 30
- Issue
- 1
- Pages
- 56-79
- ISSN
- 0278-3649 (ISSN-L)
- 1741-3176
- OpenAlex
- W2147536420
- Cited by
- 536
- References
- 51
- FWCI
- 51.34
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.