On-Manifold Preintegration for Real-Time Visual--Inertial Odometry
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Abstract
Current approaches for visual-inertial odometry (VIO) are able to attain highly accurate state estimation via nonlinear optimization. However, real-time optimization quickly becomes infeasible as the trajectory grows over time; this problem is further emphasized by the fact that inertial measurements come at high rate, hence, leading to the fast growth of the number of variables in the optimization. In this paper, we address this issue by preintegrating inertial measurements between selected keyframes into single relative motion constraints. Our first contribution is a preintegration theory that properly addresses the manifold structure of the rotation group. We formally discuss the generative measurement model as well as the nature of the rotation noise and derive the expression for the maximum a posteriori state estimator. Our theoretical development enables the computation of all necessary Jacobians for the optimization and a posteriori bias correction in analytic form. The second contribution is to show that the preintegrated inertial measurement unit model can be seamlessly integrated into a visual-inertial pipeline under the unifying framework of factor graphs. This enables the application of incremental-smoothing algorithms and the use of a structureless model for visual measurements, which avoids optimizing over the 3-D points, further accelerating the computation. We perform an extensive evaluation of our monocular VIO pipeline on real and simulated datasets. The results confirm that our modeling effort leads to an accurate state estimation in real time, outperforming state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 2×
“We use the relative error metrics proposed in [68] to obtain error statistics.”
MethodEstimation with Applications to Tracking and Navigation: Theory, Algorithms and Software
Bar‐Shalom, Y., Li, X. ‐., & Kirubarajan, T. (2002) · doi:10.1002/0471221279 · Cited by 6,782
A Mathematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. S. (2017) · CRC Press eBooks · doi:10.1201/9781315136370 · Cited by 6,694 · cited 1×
“For this purpose, we introduce the following kinematic model [49], [53]:”
MethodFactor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 1×
“This naturally leads to the well known factor graph representation, a class of bipartite graphical models that can be used to represent such factored densities [51], [52].”
BackgroundParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 1×
“A breakthrough has been the development of incremental smoothing techniques (iSAM [36], iSAM2 [3]), which leverage the expressiveness of factor graphs to maintain sparsity and to identify and update only the typically small subset of variables affected by a new measurement.”
MethodMonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · cited 1×
“The complexity of the EKF grows quadratically in the number of estimated landmarks, therefore, a small number of landmarks (in the order of 20) are typically tracked to allow real-time operation [10–12].”
BackgroundMatrix Analysis and Applied Linear Algebra
Meyer, C. (2000) · doi:10.1137/1.9780898719512 · Cited by 3,565 · cited 1×
“Any basis El ∈ R2nl×2nl−3 of the null space of El satisfies the following relation [71]:”
BackgroundMatrix analysis and applied linear algebra
(2001) · Choice Reviews Online · doi:10.5860/choice.38-5065 · Cited by 3,300
Stochastic models, estimation, and control
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6186-7 · Cited by 3,229
Cited by
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
Qin, T., Li, P., & Shen, S. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2853729 · Cited by 4,669
“We note that our current IMU preintegration procedure shares almost the same numerical results as [19] and [24], but using different derivations.”
MethodResultInfluentialORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM
Campos, C., Elvira, R., Rodriguez, J. J. G., M. Montiel, J. M., & D. Tardos, J. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3075644 · Cited by 4,274
Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., & Leonard, J. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2624754 · Cited by 3,596
“et of linear equations (the so called normal equations). These approaches can be seamlessly generalized to variables belonging to smooth manifolds (e.g., rotations), which are of interest in robotics [1, 86]. The key insight behind modern SLAM solvers is that the matrix appearing in the normal equations is sparse and…”
Self-citationPast, present, and future of simultaneous localization and mapping: Toward the robust-perception age
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-138892 · Cited by 3,048
Self-citationLIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping
Shan, T., Englot, B., Meyers, D., Wang, W., Ratti, C., & Rus, D. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341176 · Cited by 2,199
“Due to space limitations, we refer the reader to the description from [20] for the detailed derivation of Eqs.”
MethodShintake, J., Cacucciolo, V., Floreano, D., & Shea, H. (2018) · Advanced Materials · doi:10.1002/adma.201707035 · Cited by 1,938
FAST-LIO2: Fast Direct LiDAR-Inertial Odometry
Xu, W., Cai, Y., He, D., Lin, J., & Zhang, F. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3141876 · Cited by 1,750
FAST-LIO: A Fast, Robust LiDAR-Inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter
Xu, W., & Zhang, F. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3064227 · Cited by 1,127
“[15] use a graph optimization with IMU pre-integration constrains [16] and plane constraints [17] from LiDAR feature points.”
MethodVisual-Inertial Monocular SLAM With Map Reuse
Mur-Artal, R., & Tardos, J. D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2653359 · Cited by 795
“Both preintegrations and Jacobians can be efficiently computed iteratively as IMU measurements arrive [7].”
BackgroundMethodInfluentialA Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry
Zhang, Z., & Scaramuzza, D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593941 · Cited by 658
“In this paper, we use the preintegrated IMU measurements proposed in [9], [10].”
BackgroundMethodSelf-citation(2020) · doi:10.1109/icra40945.2020.9196885 · Cited by 633
Self-citationVidal, A. R., Rebecq, H., Horstschaefer, T., & Scaramuzza, D. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2793357 · Cited by 541
“A recent approach in this category was proposed by [3].”
BackgroundMethodInfluentialSelf-citationIterated extended Kalman filter based visual-inertial odometry using direct photometric feedback
Bloesch, M., Burri, M., Omari, S., Hutter, M., & Siegwart, R. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917728574 · Cited by 485
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438
“25 SVO [22] 0.”
Events-To-Video: Bringing Modern Computer Vision to Event Cameras
Rebecq, H., Ranftl, R., Koltun, V., & Scaramuzza, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00398 · Cited by 420
“Because of its importance in augmented/virtual reality and mobile robotics, VIO has been extensively studied in the last decade and is relatively mature today [28, 23, 6, 13, 36].”
BackgroundSelf-citationA Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
Delmerico, J., & Scaramuzza, D. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460664 · Cited by 399
“The experiments in [8] compare the SVO + GTSAM system to OKVIS and a non-public implementation of MSCKF, but only on non-public datasets without flyingrobot-like motions.”
MethodSelf-citationGVINS: Tightly Coupled GNSS–Visual–Inertial Fusion for Smooth and Consistent State Estimation
Cao, S., Lu, X., & Shen, S. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3133730 · Cited by 406
Visual-Inertial Navigation: A Concise Review
Huang, G. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793604 · Cited by 391
“was introduced in [29, 34] which presents a singularity-free”
BackgroundUsing Inertial Sensors for Position and Orientation Estimation
Kok, M., Hol, J. D., & Schön, T. B. (2017) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000094 · Cited by 347
“As argued in [38], it is more intuitive to directly represent the uncertainty in terms of orientation deviations.”
ESIM: an Open Event Camera Simulator
Rebecq, H., Gehrig, D., & Scaramuzza, D. (2018) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-175993 · Cited by 194
Self-citationPL-VIO: Tightly-Coupled Monocular Visual–Inertial Odometry Using Point and Line Features
He, Y., Zhao, J., Guo, Y., He, W., & Yuan, K. (2018) · Sensors · doi:10.3390/s18041159 · Cited by 330
“[14] reformulated the IMU pre-integration by treating the rotation group on a manifold instead of using Euler angles.”
BackgroundMethodInfluentialContact-aided invariant extended Kalman filtering for robot state estimation
Hartley, R., Ghaffari, M., Eustice, R. M., & Grizzle, J. W. (2020) · The International Journal of Robotics Research · doi:10.1177/0278364919894385 · Cited by 321
“The theory of invariant observer design is based on the estimation error being invariant under the action of a matrix Lie group (Aghannan and Rouchon, 2002; Bonnabel et al., 2009), which has recently led to the development of the InEKF1 (Barrau, 2015; Barrau and Bonnabel, 2017, 2018; Bonnabel, 2007) with successful ap…”
BackgroundMethodInfluentialA General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors
Qin, T., Pan, J., Cao, S., & Shen, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.03638 · Cited by 279
“2) IMU Factor: We use the well-known IMU preinte-gration algorithm [11, 12] to construct the IMU factor.”
MethodRoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, & New Methods
Herath, S., Yan, H., & Furukawa, Y. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196860 · Cited by 302
“IMU sensor fusion produces real-time orientations of Virtual Reality headsets, augments SLAM [1]–[3] in Augmented Reality [4]–[6], and allows enhanced navigation and control in many domains.”
BackgroundA Review of Multi-Sensor Fusion SLAM Systems Based on 3D LIDAR
Xu, X., Zhang, L., Yang, J., Cao, C., Wang, W., Ran, Y., Tan, Z., & Luo, M. (2022) · Remote Sensing · doi:10.3390/rs14122835 · Cited by 302
“For the specific derivation of residuals, please refer to the literature [53,54].”
BackgroundKimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems
Tian, Y., Chang, Y., Herrera Arias, F., Nieto-Granda, C., How, J. P., & Carlone, L. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3137751 · Cited by 288
Self-citationKimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211056674 · Cited by 271
“preintegration and fixed-lag smoothing (Forster et al. 2017), (ii) Kimera-PGMO: a robust pose graph and mesh optimizer, that generalizes Kimera-RPGO, which only optimized the pose graph.”
BackgroundMethodInfluentialSelf-citationOnline Temporal Calibration for Monocular Visual-Inertial Systems
Qin, T., & Shen, S. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593603 · Cited by 231
Aslan, M. F., Durdu, A., Sabanci, K., Ropelewska, E., & Gültekin, S. S. (2022) · Applied Sciences · doi:10.3390/app12031047 · Cited by 233
“The methods used were multi-state constraint Kalman filter (MSCKF) [110], open keyframe-based visual-inertial SLAM (OKVIS) [111], robust visual-inertial odometry (ROVIO) [112], monocular visual-inertial system (VINS-Mono) [108], semi-direct visual odometry (SVO) [113] + multisensor fusion (MSF) [114], (SVO-MSF) [115] a…”
Present and Future of SLAM in Extreme Environments: The DARPA SubT Challenge
Ebadi, K., Bernreiter, L., Biggie, H., Catt, G., Chang, Y., Chatterjee, A., Denniston, C. E., Deschênes, S.-P., Harlow, K., Khattak, S., et al. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2023.3323938 · Cited by 212
“The factor graph is implemented using GTSAM [72] and integrates relative LO estimates with IMU preintegration factors [73].”
BackgroundMethodSelf-citationA Survey on Odometry for Autonomous Navigation Systems
Mohamed, S. A. S., Haghbayan, M.-H., Westerlund, T., Heikkonen, J., Tenhunen, H., & Plosila, J. (2019) · IEEE Access · doi:10.1109/access.2019.2929133 · Cited by 214
“To address these issues, in [136], the authors proposed a VIO system based on an incremental-smoothing framework and a preintegration theory.”
BackgroundContinuous-Time Visual-Inertial Odometry for Event Cameras
Mueggler, E., Gallego, G., Rebecq, H., & Scaramuzza, D. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2858287 · Cited by 193
“These methods operate by processing small groups of events from which features (such as Harris [29] or FAST [30], [31]) can be tracked and fed into standard geometric VIO algorithms (MSCKF [32], OKVIS [33], optimization with preintegrated IMU factors [34]).”
MethodSelf-citationPoint‐LIO: Robust High‐Bandwidth Light Detection and Ranging Inertial Odometry
He, D., Xu, W., Chen, N., Kong, F., Yuan, C., & Zhang, F. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202200459 · Cited by 178
VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots
Wisth, D., Camurri, M., & Fallon, M. (2023) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3193788 · Cited by 178
Direct LiDAR-Inertial Odometry: Lightweight LIO with Continuous-Time Motion Correction
Chen, K., Nemiroff, R., & Lopez, B. T. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160508 · Cited by 173
“integration [29], which is subsequently refined by solving a set of analytical continuous-time equations in W (Fig.”
Selective Sensor Fusion for Neural Visual-Inertial Odometry
Chen, C., Rosa, S., Miao, Y., Lu, C. X., Wu, W., Markham, A., & Trigoni, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01079 · Cited by 168
Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude Estimation
Brossard, M., Bonnabel, S., & Barrau, A. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3003256 · Cited by 237
“Future work will address new challenges in three directions: learning from multiple IMUs (the current method is reserved for one IMU only which serves for training and testing); learning from moderately accurate ground truth that can be output of visual-inertial localization systems; and denoising accelerometers based…”
BackgroundAVP-SLAM: Semantic Visual Mapping and Localization for Autonomous Vehicles in the Parking Lot
Qin, T., Chen, T., Chen, Y., & Su, Q. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340939 · Cited by 188
“Popular relative localization includes visual odometry [1, 2, 9]–[11], visual-inertial odometry [3]– [5, 12], Lidar odometry [6]–[8].”
BackgroundKaufmann, E., Loquercio, A., Ranftl, R., Müller, M., Koltun, V., & Scaramuzza, D. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.040 · Cited by 222
Self-citationConvergence and Consistency Analysis for a 3-D Invariant-EKF SLAM
Zhang, T., Wu, K., Song, J., Huang, S., & Dissanayake, G. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2651376 · Cited by 158
“, but under a restrictive assumption that the Jacobians are evaluated at the ground truth. Recently, Lie group representation for three-dimensional orientation/pose has become popular in SLAM. (e.g., [13][14]), which can achieve better convergence and accuracy for both filter based algorithms (e.g., [15][16]) and the o…”
MethodA micro Lie theory for state estimation in roboticsPreprint
Sol\`a, J., Deray, J., & Atchuthan, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.01537 · Cited by 751
Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
Tang, Y. S., Zhao, C., Wang, J., Zhang, C., Sun, Q., Zheng, W. X., Du, W., Qian, F., & Kurths, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3167688 · Cited by 199
ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM
Liu, H., Chen, M., Zhang, G., Bao, H., & Bao, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00211 · Cited by 170
Unified Multi-Modal Landmark Tracking for Tightly Coupled Lidar-Visual-Inertial Odometry
Wisth, D., Camurri, M., Das, S., & Fallon, M. F. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3056380 · Cited by 167
“For the definition of the residuals, see [22].”
BackgroundGNSS/INS/LiDAR-SLAM Integrated Navigation System Based on Graph Optimization
Chang, L., Niu, X., Liu, T., Tang, J., & Qian, C. (2019) · Remote Sensing · doi:10.3390/rs11091009 · Cited by 150
“In the back-end, the GNSS position provides the absolute position constraint, while the IMU preintegration [9,23] is applied to increase the motion constraints, and the LiDAR-SLAM scan matching provides the relative pose constraints.”
MethodCVI-SLAM—Collaborative Visual-Inertial SLAM
Karrer, M., Schmuck, P., & Chli, M. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2837226 · Cited by 162
“For a detailed explanation of the used pre-integration method, we kindly refer the reader to [5].”
MethodSurvey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality
Li, J., Yang, B., Danpeng, C., Wang, N., Guofeng, Z., & Bao, H. (2019) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2019.07.002 · Cited by 151
“Another method for fusing the inertial measurements into VSLAM is to summarize a group of sequential IMU readings into one single pre-integrated IMU measurements, making it convenient to incorporate the bias updates during optimization, such as [12, 13].”
MethodDeep Learning for Inertial Positioning: A Survey
Chen, C., & Pan, X. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3381161 · Cited by 184
“Traditional VIO methods integrate visual and inertial information based on filtering [8], [88], fixed-lag smoothing [89], or full smoothing [90].”
MethodCvišić, I., Ćesić, J., Marković, I., & Petrović, I. (2018) · Journal of Field Robotics · doi:10.1002/rob.21762 · Cited by 133
“Odometry based on motion integration is prone to drift, although recent methods achieve very accurate performance both with 7,8 and without inertial information.”
MethodRange-Focused Fusion of Camera-IMU-UWB for Accurate and Drift-Reduced Localization
Nguyen, T. H., Nguyen, T., & Xie, L. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3057838 · Cited by 137
“The IMU pre-integration methods [32] allow the highfrequency linear acceleration and angular velocity to be pre-”
MethodSVIn2: An Underwater SLAM System using Sonar, Visual, Inertial, and Depth Sensor
Rahman, S., Li, A. Q., & Rekleitis, I. M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967703 · Cited by 170
“The transformation from minimal coordinates to tangent space can be done using a bijective mapping [9], [36]:”
MethodEvent-aided Direct Sparse Odometry
Hidalgo‐Carrió, J., Gallego, G., & Scaramuzza, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00569 · Cited by 179
“2), EDS outperforms all other monocular baseline methods, even without using inertial measurements (which are known to improve robustness and increase accuracy in VO [57]).”
BackgroundSelf-citationReview on LiDAR-based SLAM Techniques
Huang, L. (2021) · doi:10.1109/conf-spml54095.2021.00040 · Cited by 164
AlphaPilot: Autonomous Drone Racing
Foehn, P., Brescianini, D., Kaufmann, E., Cieslewski, T., Gehrig, M., Muglikar, M., & Scaramuzza, D. (2020) · Autonomous Robots · doi:10.1007/s10514-021-10011-y · Cited by 203
“To apply the EKFs linear update step on the over-parameterized quaternion, it is lifted to its tangent space description, similar to [33].”
MethodSelf-citationCamera, LiDAR, and IMU Based Multi-Sensor Fusion SLAM: A Survey
Zhu, J., Li, H., & Zhang, T. (2023) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010010 · Cited by 163
Advances in intelligent and autonomous navigation systems for small UAS
Bijjahalli, S., Sabatini, R., & Gardi, A. (2020) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2020.100617 · Cited by 123
UV-SLAM: Unconstrained Line-Based SLAM Using Vanishing Points for Structural Mapping
Lim, H., Jeon, J., & Myung, H. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3140816 · Cited by 134
Han, L., Lin, Y., Du, G., & Lian, S. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968467 · Cited by 125
“status of IMU is updated as follows[6], [32]:”
BackgroundConsistent EKF-Based Visual-Inertial Odometry on Matrix Lie Group
Heo, S., & Park, C. G. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2808330 · Cited by 143
“[20] proposed IMU-preintegration method on manifold that deals with the preintegrated IMU measurement error in a probabilistic manner to achieve better convergence and accuracy.”
MethodDynam-SLAM: An Accurate, Robust Stereo Visual-Inertial SLAM Method in Dynamic Environments
Yin, H., Li, S., Tao, Y., Guo, J., & Huang, B. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3199087 · Cited by 130
“The state error transmission in (17) is divided into two parts [43]: the state error at the current time is transmitted to the next time, and the state measurement noise at the current time is also transmitted to the next time.”
BackgroundMethodA Light-Weight Semantic Map for Visual Localization towards Autonomous Driving
Qin, T., Zheng, Y., Chen, T., Chen, Y., & Su, Q. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561663 · Cited by 160
“Popular approaches include camera-only methods [1]–[5] and visual-inertial methods [6]–[9].”
MethodImage-based camera localization: an overview
Wu, Y., Tang, F., & Li, H. (2018) · Visual Computing for Industry Biomedicine and Art · doi:10.1186/s42492-018-0008-z · Cited by 132
“Forster et al. [117] presented a preintegration theory that appropriately addresses the manifold structure of the rotation group.”
BackgroundVIRAL-Fusion: A Visual-Inertial-Ranging-Lidar Sensor Fusion Approach
Nguyen, T., Cao, M., Yuan, S., Lyu, Y., Nguyen, T. H., & Xie, L. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3094157 · Cited by 124
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12567 · Cited by 207
“…and algorithms being developed, for example, odometry estimation (including visual odometry [9], [10], [11], visual-inertial odometry [12], [13], [14], [15] and LIDAR odometry [16]), image-based localization [17], [18], place recognition [19], SLAM [10], [20], [21], and structure from motion…”
MethodSVIn2: A multi-sensor fusion-based underwater SLAM system
Rahman, S., Li, A. Q., & Rekleitis, I. M. (2022) · The International Journal of Robotics Research · doi:10.1177/02783649221110259 · Cited by 143
“In recent years, many visual odometry and SLAM systems (Davison et al., 2007; Engel et al., 2014, 2018; Forster et al., 2017b; Klein and Murray, 2007; Mur-Artal et al., 2015) have been developed using monocular, stereo, or multi-camera rigs mostly for indoor and outdoor environments that can be…”
BackgroundMethodImpact of the Earth Rotation Compensation on MEMS-IMU Preintegration of Factor Graph Optimization
Tang, H., Zhang, T., Niu, X., Fan, J. F., & Liu, J. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3192552 · Cited by 113
“The noise covariance strongly influences the MAP estimator because the inverse noise covariance (information matrix) is used to weight the factor in the FGO [18], [23].”
InfluentialAttention and Anticipation in Fast Visual-Inertial Navigation
Carlone, L., & Karaman, S. (2017) · IEEE Transactions on Robotics · doi:10.1109/icra.2017.7989448 · Cited by 126
Self-citationLearning Wheel Odometry and IMU Errors for Localization
Brossard, M., & Bonnabel, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794237 · Cited by 114
“Our filter follows the state delayed EKF of [18]: it uses a differential-drive process model to integrate measurements from the wheel encoders, the FoG gyro are used for propagating heading, a constant velocity model for the roll and pitch is assumed, and we adapt the IMU measurements in [18] into the form of pre-inte…”
BackgroundMethodInfluentialA Review of Small UAV Navigation System Based on Multisource Sensor Fusion
Ye, X., Song, F., Zhang, Z., & Qing-hua, Z. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3292427 · Cited by 132
COVINS: Visual-Inertial SLAM for Centralized Collaboration
Schmuck, P., Ziegler, T., Karrer, M., Perraudin, J., & Chli, M. (2021) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct54149.2021.00043 · Cited by 118
An invariant-EKF VINS algorithm for improving consistency
Wu, K., Zhang, T. L., Su, D., Huang, S., & Dissanayake, G. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205965 · Cited by 135
Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios
Camurri, M., Ramezani, M., Nobili, S., & Fallon, M. F. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.00068 · Cited by 137
“…Equations (13)–(17), we can predict the mean of the state x k by simple integration over the period 1 t = t k − t k − 1 : Note that the attitude is integrated separately using the exponential map between the Lie group of rotations and its Lie algebra at the identity (see Forster et al., 2017a).”
MethodEnabling High-Precision Visible Light Localization in Today's Buildings
Zhu, S., & Zhang, X. (2017) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081335 · Cited by 122
“State-of-the-art vision-based robotic systems integrate motion sensors with a camera to realize visual-inertial odometry (VIO) or SLAM [34], which tracks user movement continuously via image differential [42].”
BackgroundPulsar: Towards Ubiquitous Visible Light Localization
Zhang, C., & Zhang, X. (2017) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3117811.3117821 · Cited by 107
“We walk across each venue following a marked route, and then compare the location traces generated by Pulsar and by Tango’s visual-inertial odometry algorithm (shown to have sub-meter precision [16]).”
BackgroundMethodInfluentialCollaborative visual SLAM for multiple agents:A brief survey
Zou, D., Tan, P., & Yu, W. (2019) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2019.09.002 · Cited by 99
MIMC-VINS: A Versatile and Resilient Multi-IMU Multi-Camera Visual-Inertial Navigation System
Eckenhoff, K., Geneva, P., & Huang, G. P. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3049445 · Cited by 114
State of the Art in Vision-Based Localization Techniques for Autonomous Navigation Systems
Alkendi, Y., Seneviratne, L. D., & Zweiri, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3082778 · Cited by 125
“IMU preintegration process was reformulated in Forster et al. [146] by using the rotation group which was computed by a manifold rather than by Euler angles.”
BackgroundMethodRobust Real-time LiDAR-inertial Initialization
Zhu, F., Ren, Y., & Zhang, F. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982225 · Cited by 99
“[5, 6], LiDAR-inertial system [1, 7, 8], and LiDAR inertial visual system [9, 10].”
Cioffi, G., & Scaramuzza, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341697 · Cited by 118
“We use the IMU preintegration derivation proposed in [17], which is based on the continuoustime quaternion-based formulation in [18] and includes the manipulation of IMU biases as in [7].”
InfluentialSelf-citationGR-LOAM: LiDAR-based sensor fusion SLAM for ground robots on complex terrain
Su, Y., Wang, T., Shao, S., Yao, C., & Wang, Z. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103759 · Cited by 109
Closed-form preintegration methods for graph-based visual–inertial navigation
Eckenhoff, K., Geneva, P., & Huang, G. P. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919835021 · Cited by 118
RGBD-Inertial Trajectory Estimation and Mapping for Ground Robots
Shan, Z., Li, R., & Schwertfeger, S. (2019) · Sensors · doi:10.3390/s19102251 · Cited by 108
“The bias is usually modeled as a random walk process [15,17,59].”
BackgroundMixed reality and remote sensing application of unmanned aerial vehicle in fire and smoke detection
Esfahlani, S. S. (2019) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2019.04.006 · Cited by 110
Deep Neural Network Based Inertial Odometry Using Low-Cost Inertial Measurement Units
Chen, C., Lu, C. X., Wahlström, J., Markham, A., & Trigoni, N. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2960780 · Cited by 106
“The effective fusion of inertial sensors and cameras can be achieved via extended kalman filtering (EKF) [13], or graph optimization [14], [15], [16], where the raw visual and inertial”
MethodIntermittent GPS-aided VIO: Online Initialization and Calibration
Lee, W., Eckenhoff, K., Geneva, P., & Huang, G. P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197029 · Cited by 112
iG-LIO: An Incremental GICP-Based Tightly-Coupled LiDAR-Inertial Odometry
Chen, Z., Xu, Y. K., Yuan, S., & Xie, L. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3349915 · Cited by 108
Gaussians on Riemannian Manifolds: Applications for Robot Learning and Adaptive Control
Calinon, S. (2020) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2020.2980548 · Cited by 117
“For example, in [15], the manifold structure of the rotation group is exploited for preintegration and uncertainty propagation in SO(3).”
BackgroundFactor Graphs: Exploiting Structure in Robotics
Dellaert, F. (2021) · Annual Review of Control Robotics and Autonomous Systems · doi:10.1146/annurev-control-061520-010504 · Cited by 129
Self-citationVIR-SLAM: visual, inertial, and ranging SLAM for single and multi-robot systems
Cao, Y., & Beltrame, G. (2021) · Autonomous Robots · doi:10.1007/s10514-021-09992-7 · Cited by 105
Sonar Visual Inertial SLAM of Underwater Structures
Rahman, S., Li, A. Q., & Rekleitis, I. M. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460545 · Cited by 106
“which represents the error for each component of the state vector with a transformation between tangent space and minimal coordinates [41].”
BackgroundYan, Y., Zhang, B., Zhou, J. C., Zhang, Y., & Liu, X. (2022) · Agronomy · doi:10.3390/agronomy12081740 · Cited by 110
Associating Uncertainty to Extended Poses for on Lie Group IMU Preintegration With Rotating Earth
Brossard, M., Barrau, A., Chauchat, P., & Bonnabel, S. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3100156 · Cited by 88
Robust Legged Robot State Estimation Using Factor Graph Optimization
Wisth, D., Camurri, M., & Fallon, M. F. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2933768 · Cited by 87
“where the covariance ΣQΔi is provided by the filter, and Φ is the lifting operator defined in [17].”
BackgroundMethodInfluentialSolid-State LiDAR based-SLAM: A Concise Review and Application
Nam, D. V., & Kim, G.-W. (2021) · IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp51126.2021.00064 · Cited by 93
“Next, the robot state is estimated by handling scan matching to a local map, and we combine IMU pre-integration [19] with doing local FOG in a sliding window.”
MethodAirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM System
Xu, K., Hao, Y., Yuan, S., Wang, C., & Xie, L. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3539171 · Cited by 142
“In the meantime, if an IMU is accessible, its measurements will be processed using the IMU preintegration method [72], and added to the initial pose estimation and local bundle adjustment.”
Gong, L., Gao, B., Sun, Y., Zhang, W., Lin, G., Zhang, Z., Li, Y., & Liu, C. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3361092 · Cited by 91
Tightly coupled integrated navigation system via factor graph for UAV indoor localization
Yang, S. S., & Hsu, L. (2020) · Aerospace Science and Technology · doi:10.1016/j.ast.2020.106370 · Cited by 89
Invariant Kalman Filtering for Visual Inertial SLAM
Brossard, M., Bonnabel, S., & Barrau, A. (2018) · International Conference on Information Fusion (FUSION) · doi:10.23919/icif.2018.8455807 · Cited by 83
“In this paper, we implemented the method of pre-integration on manifolds of [8].”
MethodLiDAR Inertial Odometry Aided Robust LiDAR Localization System in Changing City Scenes
Ding, W., Hou, S., Gao, H., Wan, G., & Song, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196698 · Cited by 87
“Please refer to [30] for details.”
MethodInfluentialDegenerate Motion Analysis for Aided INS With Online Spatial and Temporal Sensor Calibration
Yang, Y., Geneva, P., Eckenhoff, K., & Huang, G. P. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2893803 · Cited by 95
Metadata
- Authors
- Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D.
- Affiliations
- Massachusetts Institute of Technology (US)
- Georgia Institute of Technology (US)
- University of Zurich (CH)
- Venue
- IEEE Transactions on Robotics
- Published
- 2017-02
- Volume
- 33
- Issue
- 1
- Pages
- 1-21
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W2482726005
- Cited by
- 1,502
- References
- 83
- Citations by year
- 2026: 156 · 2025: 179 · 2024: 234 · 2023: 193 · 2022: 180 · 2021: 174 · 2020: 134 · 2019: 129 · 2018: 88 · 2017: 32
- FWCI
- 102.35
- Type
- journal article
- Language
- en
- Open access
- green
- www.zora.uzh.ch
- Licence
- Locations
- IEEE Transactions on Robotics (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Swiss National Foundation (200021-143607)
- National Center of Competence in Research Robotics (NCCR)
- University of Zurich
- National Science Foundation (11115678)
- United States Department of Agriculture (USDA)
- National Institute of Food and Agriculture (GEOW-2014-09160)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Aerospace Engineering › Inertial Sensor and Navigation
- Physical Sciences › Computer Science › Artificial Intelligence › Target Tracking and Data Fusion in Sensor Networks
- Keywords
- visual-inertial odometry, inertial measurement unit, factor graphs, maximum a posteriori estimation, bias correction, real-time state estimation, keyframe selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.