On-Manifold Preintegration for Real-Time Visual--Inertial Odometry
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Abstract
Current approaches for visual-inertial odometry (VIO) are able to attain highly accurate state estimation via nonlinear optimization. However, real-time optimization quickly becomes infeasible as the trajectory grows over time; this problem is further emphasized by the fact that inertial measurements come at high rate, hence, leading to the fast growth of the number of variables in the optimization. In this paper, we address this issue by preintegrating inertial measurements between selected keyframes into single relative motion constraints. Our first contribution is a preintegration theory that properly addresses the manifold structure of the rotation group. We formally discuss the generative measurement model as well as the nature of the rotation noise and derive the expression for the maximum a posteriori state estimator. Our theoretical development enables the computation of all necessary Jacobians for the optimization and a posteriori bias correction in analytic form. The second contribution is to show that the preintegrated inertial measurement unit model can be seamlessly integrated into a visual-inertial pipeline under the unifying framework of factor graphs. This enables the application of incremental-smoothing algorithms and the use of a structureless model for visual measurements, which avoids optimizing over the 3-D points, further accelerating the computation. We perform an extensive evaluation of our monocular VIO pipeline on real and simulated datasets. The results confirm that our modeling effort leads to an accurate state estimation in real time, outperforming state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 2×
“We use the relative error metrics proposed in [68] to obtain error statistics.”
MethodEstimation with Applications to Tracking and Navigation: Theory, Algorithms and Software
Bar‐Shalom, Y., Li, X. ‐., & Kirubarajan, T. (2002) · doi:10.1002/0471221279 · Cited by 6,782
A Mathematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. S. (2017) · CRC Press eBooks · doi:10.1201/9781315136370 · Cited by 6,694 · cited 1×
“For this purpose, we introduce the following kinematic model [49], [53]:”
MethodFactor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 1×
“This naturally leads to the well known factor graph representation, a class of bipartite graphical models that can be used to represent such factored densities [51], [52].”
BackgroundParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 1×
“A breakthrough has been the development of incremental smoothing techniques (iSAM [36], iSAM2 [3]), which leverage the expressiveness of factor graphs to maintain sparsity and to identify and update only the typically small subset of variables affected by a new measurement.”
MethodMonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · cited 1×
“The complexity of the EKF grows quadratically in the number of estimated landmarks, therefore, a small number of landmarks (in the order of 20) are typically tracked to allow real-time operation [10–12].”
BackgroundMatrix Analysis and Applied Linear Algebra
Meyer, C. (2000) · doi:10.1137/1.9780898719512 · Cited by 3,565 · cited 1×
“Any basis El ∈ R2nl×2nl−3 of the null space of El satisfies the following relation [71]:”
BackgroundMatrix analysis and applied linear algebra
(2001) · Choice Reviews Online · doi:10.5860/choice.38-5065 · Cited by 3,300
Stochastic models, estimation, and control
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6186-7 · Cited by 3,229
Cited by
Items 201–300 of 1,000
Factor Graphs for Navigation Applications: A Tutorial
Taylor, C. N., & Gross, J. N. (2024) · NAVIGATION Journal of the Institute of Navigation · doi:10.33012/navi.653 · Cited by 50
Preintegrated Velocity Bias Estimation to Overcome Contact Nonlinearities in Legged Robot Odometry
Wisth, D., Camurri, M., & Fallon, M. F. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197214 · Cited by 53
“(24) where Jr is the right Jacobian of SO(3) and the other terms are manipulations of δφ and δp from [6] and Eq.”
BackgroundMethodInfluentialROVINS: Robust Omnidirectional Visual Inertial Navigation System
Seok, H., & Lim, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3010457 · Cited by 46
“The IMU motion between two consecutive keyframes can be defined in terms of the pre-integration ΔR, Δv, and Δp from all measurements in-between, and the IMU orientations wR ∈ SO(3), position wp, and velocity b wv, as described in [15]:”
BackgroundMethodInfluentialA Right Invariant Extended Kalman Filter for Object Based SLAM
Song, Y., Zhang, Z., Wu, J., Wang, Y., Zhao, L., & Huang, S. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3139370 · Cited by 51
“Exploiting the properties of invariant tangent vector field, Lie group theory generates the invariant-EKF methodology [23], [24], [25], which is firstly applied to EKF-SLAM in [25].”
Doer, C., & Trommer, G. F. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636799 · Cited by 55
“Our main contributions are [1], [2], [7]–[10] or fixed lag smoothing [3], [11], [12].”
BackgroundGlobally Consistent and Tightly Coupled 3D LiDAR Inertial Mapping
Koide, K., Yokozuka, M., Oishi, S., & Banno, A. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812385 · Cited by 42
A Robust LiDAR SLAM Method for Underground Coal Mine Robot with Degenerated Scene Compensation
Yang, X., Lin, X., Yao, W., Ma, H., Zheng, J., & Ma, B. (2022) · Remote Sensing · doi:10.3390/rs15010186 · Cited by 49
Piao, J.-C., & Kim, S. (2017) · Sensors · doi:10.3390/s17112567 · Cited by 47
“…the two frames, that As shown in the above formula in [32], the constraints between the two frames can be expressed using only IMU data; furthermore, they can be expressed according to the states of the two frames. increments between the two frames can be obtained by solving the above equation.”
BackgroundMethodInfluentialWen, S., Zhao, Y., Liu, X., Sun, F., Lu, H., & Wang, Z. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3041852 · Cited by 41
“The IMU module is fused with a camera which can jointly optimize the raw measurement [22], [23].”
MethodA Comparison of Graph Optimization Approaches for Pose Estimation in SLAM
Jurić, A., Kendes, F., Marković, I., & Petrovic, I. K. (2021) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.23919/mipro52101.2021.9596721 · Cited by 50
“ORB-SLAM ( [16], [17]) uses g2o as a back-end for camera pose optimization, and SVO [18] uses it for visual odometry.”
MethodExperimental evaluation of Visual‐Inertial Odometry systems for arable farming
Cremona, J., Comelli, R., & Pire, T. (2022) · Journal of Field Robotics · doi:10.1002/rob.22099 · Cited by 47
“SVO 2.0 SVO 2.0 (Forster et al., 2017b) is an extension of the original SVO (Semi-direct Visual Odometry) implementation presented by Forster et al. (2014).”
InfluentialWu, W., Zhong, X., Wu, D., Chen, B., Zhong, X., & Liu, Q. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3240372 · Cited by 40
“In addition, the relative transformation ΔTk,k+1 expressed in (2) is used to construct the LiDAR odometry residual model rL(m,X) [18].”
BackgroundMethodPIVO: Probabilistic Inertial-Visual Odometry for Occlusion-Robust Navigation
Solin, A., Cortés, S., Rahtu, E., & Kannala, J. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00073 · Cited by 43
Visual-Inertial Navigation Algorithm Development Using Photorealistic Camera Simulation in the Loop
Sayre-McCord, T., Guerra, W., Antonini, A., Arneberg, J., Brown, A., Cavalheiro, G. V., Fang, Y., Gorodetsky, A., Mccoy, D. S., Quilter, S., et al. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460692 · Cited by 47
“Our VIO approach is based on the work by Forster et al. [24] with modifications made to allow for real time state estimation on the limited computation of a TX1 module.”
BackgroundMethodInfluentialSelf-citationEnvironment Driven Underwater Camera-IMU Calibration for Monocular Visual-Inertial SLAM
Gu, C., Cong, Y., & Sun, G. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793577 · Cited by 40
“In addition, IMU position αk bk+1 , velocity β bk bk+1 and the angle between the two body frame bk and bk+1 can be measured by continuous-time quaternion-based derivation of IMU preintegration [9][29].”
BackgroundA Survey of Simultaneous Localization and Mapping.
Huang, B., Zhao, J., & Liu, J. (2019) · Cited by 53
Simultaneous Hand–Eye/Robot–World/Camera–IMU Calibration
Wu, J., Wang, M., Jiang, Y., Yi, B., Fan, R., & Liu, M. (2021) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2021.3103995 · Cited by 40
“is based on motion-based calibration concepts [18], [19] that significantly require IMU preintegration [20], which diverges”
BackgroundKachurka, V., Rault, B., Muñoz, F. I. I., Roussel, D., Bonardi, F., Didier, J.-Y., Hadj-Abdelkader, H., Bouchafa, S., Alliez, P., & Robin, M. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3101121 · Cited by 43
“The problem formulation for the visual-inertial bundle adjustment (equation and notation adopted from [53, eq.”
3D LiDAR/IMU Calibration Based on Continuous-Time Trajectory Estimation in Structured Environments
Li, S., Wang, L., Li, J., Tian, B., Chen, L. F., & Li, G. (2021) · IEEE Access · doi:10.1109/access.2021.3114618 · Cited by 43
“INITIALIZING THE INITIAL GRAVITY VECTOR Referring to [29], the translation part of the PIM can be rewritten as:”
MethodLog-linear Error State Model Derivation without Approximation for INS
Chang, L., & Luo, Y. (2022) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2022.3197726 · Cited by 37
“It should be noted that the group state decomposition is also the key point for accurate preintegration in inertial-based robotics applications [19], [21], [22], [23], [24].”
BackgroundEdge-Based Monocular Thermal-Inertial Odometry in Visually Degraded Environments
Wang, Y., Chen, H., Liu, Y., & Zhang, S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3246381 · Cited by 37
Rolling-Shutter Modelling for Direct Visual-Inertial Odometry
Schubert, D., Demmel, N., von Stumberg, L., Usenko, V., & Cremers, D. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968539 · Cited by 46
“with appropriate covariances Σ as derived in [23].”
On the Recursive Joint Position and Attitude Determination in Multi-Antenna GNSS Platforms
Medina, D., Vilà‐Valls, J., Heßelbarth, A., Ziebold, R., & García, J. (2020) · Remote Sensing · doi:10.3390/rs12121955 · Cited by 40
Franchi, M., Ridolfi, A., & Allotta, B. (2020) · Journal of Field Robotics · doi:10.1002/rob.21991 · Cited by 37
Jia, B., Wang, L., Wang, R., Jin, J., Zhao, Z., & Wu, D. (2021) · Smart Materials and Structures · doi:10.1088/1361-665x/abdc0a · Cited by 37
Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry
Zhang, L., Wisth, D., Camurri, M., & Fallon, M. F. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3137910 · Cited by 36
“We follow the standard manner of IMU measurement preintegration from [1] to constrain the pose, velocity, and biases between two consecutive nodes of the graph, providing high frequency state updates between nodes.”
InfluentialFeature-based visual odometry prior for real-time semi-dense stereo SLAM
Krombach, N., Droeschel, D., Houben, S., & Behnke, S. (2018) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.08.002 · Cited by 43
“In particular the work by Forster [15] has allowed for tight-coupling, i.e., integrating both IMU readings and visual odometry in a single non-linear cost function.”
BackgroundInfluentialVINS-Vehicle: A Tightly-Coupled Vehicle Dynamics Extension to Visual-Inertial State Estimator
Kang, R., Xiong, L., Xu, M., Zhao, J., & Zhang, P. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8916940 · Cited by 40
Constrained visual predictive control of tendon-driven continuum robots
Norouzi‐Ghazbi, S., Mehrkish, A., Fallah, M. M. H., & Janabi‐Sharifi, F. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103856 · Cited by 36
PLD-VINS: RGBD visual-inertial SLAM with point and line features
Zhu, Y., Jin, R., Lou, T.-S., & Zhao, L.-Y. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2021.107185 · Cited by 34
Aslan, M. F., Durdu, A., & Sabancı, K. (2022) · Measurement · doi:10.1016/j.measurement.2022.111030 · Cited by 40
IMU-Aided High-Frequency Lidar Odometry for Autonomous Driving
Xue, H., Fu, H., & Dai, B. (2019) · Applied Sciences · doi:10.3390/app9071506 · Cited by 38
“We introduce the following kinematic model [21]: In Equation (5), R BW τ indicates the rotation matrix from the coordinate system { B } to { W } . ω x τ , ω y τ , and ω z τ represent the instantaneous angular velocities of the IMU.”
MethodOnline IMU Intrinsic Calibration: Is It Necessary?
Yang, Y., Geneva, P., Zuo, X., & Huang, G. P. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.026 · Cited by 45
“+ GIkR̂Ξ2 Ik ã− GIkR̂Ξ4 Ik ω̃ where Jr (tk) , Jr ( Ik ω̂k∆tk ) [4] denotes the SO(3) right Jacobian.”
LSTM and Filter Based Comparison Analysis for Indoor Global Localization in UAVs
Yusefi, A., Durdu, A., Aslan, M. F., & Sungur, C. (2021) · IEEE Access · doi:10.1109/access.2021.3049896 · Cited by 37
“The traditional methods include SVOMSF [95], MSCKF [96], OKVIS [97], ROVIO [98], VINSMONO [99], VINSMONO + LC [100], SVOGTSAM [101], ORBSLAM2 [12], STCM-SLAM [102] and learning based are SelfVIO [103], Kimera [104] and methods proposed by Baldini et al. [94] and Li and Steven [105].”
MethodLIO-EKF: High Frequency LiDAR-Inertial Odometry using Extended Kalman Filters
Wu, Y., Guadagnino, T., Wiesmann, L., Klingbeil, L., Stachniss, C., & Kuhlmann, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610667 · Cited by 45
Laser Map Aided Visual Inertial Localization in Changing Environment
Ding, X., Wang, Y., Li, D., Tang, L., Yin, H., & Xiong, R. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593846 · Cited by 37
GSLAM: A General SLAM Framework and Benchmark
Zhao, Y., Xu, S., Bu, S., Jiang, H., & Han, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00120 · Cited by 41
Nam, D. V., & Kim, G.-W. (2020) · Sensors · doi:10.3390/s20102922 · Cited by 37
“The full smoothing algorithm [17–19] manipulates the whole history states of the travel pathway with their measurement constraints, then solves extensive nonlinear optimization in terms of the factor graph algorithm [12].”
MethodZhang, H., Xia, X., Nitsch, M., & Abel, D. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3189824 · Cited by 36
Path Following, Obstacle Detection and Obstacle Avoidance for Thrusted Underwater Snake Robots
Kelasidi, E., Moe, S., Pettersen, K. Y., Kohl, A. M., Liljebäck, P., & Gravdahl, J. T. (2019) · Frontiers in Robotics and AI · doi:10.3389/frobt.2019.00057 · Cited by 36
Survey of unmanned subterranean exploration, navigation, and localisation
Martz, J., Alsabban, W. H., & Smith, R. N. (2020) · IET Cyber-Systems and Robotics · doi:10.1049/iet-csr.2019.0043 · Cited by 42
“Visual methods often have high computational cost but there are solutions that are capable of real-time performance [110, 138, 139].”
BackgroundIn-Hand Object-Dynamics Inference Using Tactile Fingertips
Sundaralingam, B., & Hermans, T. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2020.3043675 · Cited by 37
RP-VIO: Robust Plane-based Visual-Inertial Odometry for Dynamic Environments
Ram, K., Kharyal, C., Harithas, S. S., & Krishna, K. M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636522 · Cited by 39
“An important enabler for optimization-based VINS is IMU preintegration [23] which allows multiple inertial measurements to be summarized, reducing state space size.”
VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation
Ding, Z., Yang, T., Zhang, K., Xu, C., & Gao, F. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560898 · Cited by 36
“External Force Preintegration We formulate our external force preintegration inspired by acceleration preintegration in [20] and [4], and extend preintegration of dynamics factor in [2] to be a dynamics and external force preintegration factor.”
MethodTightly-Coupled Visual-Inertial-Pressure Fusion Using Forward and Backward IMU Preintegration
Hu, C., Zhu, S., Liang, Y., & Song, W. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3177847 · Cited by 39
Pöppl, F., Neuner, H., Mandlburger, G., & Pfeifer, N. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.12.022 · Cited by 43
Beuchert, J., Camurri, M., & Fallon, M. F. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161522 · Cited by 36
High Speed and High Dynamic Range Video with an Event Camera
Rebecq, H., Ranftl, R., Koltun, V., & Scaramuzza, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2963386 · Cited by 50
“Because of its importance in augmented/virtual reality and mobile robotics, VIO has been extensively studied in the last decade and is relatively mature today [52], [53], [54], [55], [56].”
BackgroundSelf-citationWu, W., Li, J., Chen, C., Yang, B., Zou, X., Yang, Y., Xu, Y., Zhong, R., & Chen, R. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.04.004 · Cited by 39
PDLC-LIO: A Precise and Direct SLAM System Toward Large-Scale Environments With Loop Closures
Wang, S., Zhang, J., & Tan, X. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3307439 · Cited by 32
“…and rotational velocity of B keeps constant during the integration described above. ij , (cid:49) p ij , and (cid:49) R ij are the IMU pre-integrated measurements between time steps i and j that can be calculated using : For simplicity, the derivation of the full formula is shown in [17] and [38].”
Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning
Pan, Y., Zhou, W., Cao, Y., & Zha, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01706 · Cited by 48
“The propagation of covariance and approximate bias up date techniques are also adopted in our system, which are detailed in [ 11].”
MethodInfluentialDynamic RGB-D SLAM Based on Static Probability and Observation Number
Liu, Y., Wu, Y., & Pan, W. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3089228 · Cited by 34
Voxel-Based Localization and Mapping for Multirobot System in GPS-Denied Environments
Shen, H., Zong, Q., Tian, B., & Lu, H. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3153822 · Cited by 32
“Following the standard manner of IMU preintegration [26], the IMU prediction residual rI can be defined in the following formulation:”
MethodA GNSS/LiDAR/IMU Pose Estimation System Based on Collaborative Fusion of Factor Map and Filtering
Chen, H., Wu, W., Zhang, S., Wu, C., & Zhong, R. (2023) · Remote Sensing · doi:10.3390/rs15030790 · Cited by 37
VIRAL SLAM: Tightly Coupled Camera-IMU-UWB-Lidar SLAMPreprint
Nguyen, T., Yuan, S., Cao, M., Nguyen, T. H., & Xie, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03296 · Cited by 61
“For IMU, we employ the preintegration technique [24] to provide observation that couples two consecutive states in the sliding window.”
MethodRIO: Rotation-equivariance supervised learning of robust inertial odometry
Cao, X., Zhou, C., Zeng, D., & Wang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00650 · Cited by 48
“Unlike Visual-Inertial Odometry (VIO) [9] that is sensitive to surroundings and cannot work under extreme lighting, IMU-only inertial odometry is more desired and possible to perform accurate and robust localization every time and everywhere [11, 20].”
BackgroundAn overview of simultaneous localisation and mapping: towards multi-sensor fusion
Yin, J., Yan, F., Liu, Y., He, G., & Zhuang, Y. (2023) · International Journal of Systems Science · doi:10.1080/00207721.2023.2282409 · Cited by 40
“An inertial measurement unit (IMU) sensor measures angular and acceleration velocity data of the sensing platform at a high frequency, which can be processed as a relative motion by the preinte-gration technique (Lupton & Sukkarieh, 2012; Forster et al., 2017).”
MethodNull-space-based marginalization: Analysis and algorithm
Yang, Y., Maley, J., & Huang, G. P. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206592 · Cited by 37
“For example, in visual-inertial navigation systems (VINS) [1, 2, 3], visual point features are continuously marginalized by the Schur complement to prevent the state vector from growing too large while retaining relative camera motion constraints, which are then fused with (preintegrated) IMU measurements.”
MethodInfluentialOnline IMU Self-Calibration for Visual-Inertial Systems
Xiao, Y., Ruan, X., Chai, J., Zhang, X., & Zhu, X. (2019) · Sensors · doi:10.3390/s19071624 · Cited by 34
“This is a nontrivial extension as it introduces 24 more parameters into the state space, which makes the equations more complex compared to the general ESKF and preintegration methods showed in [8,37,38].”
MethodLow-Cost GPS-Aided LiDAR State Estimation and Map Building
Zheng, L., Zhu, Y., Xue, B., Liu, M., & Fan, R. (2019) · IEEE International Conference on Imaging Systems and Techniques (IST) · doi:10.1109/ist48021.2019.9010530 · Cited by 33
“To avoid repeatedly integrating the measurement when solving the non-linear optimization problem, we adopt the IMU preintegration method stated in [29] and [30] into our system.”
MethodA Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless NetworksPreprint
Huang, B., Zhao, J., & Liu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05214 · Cited by 31
“[70] gives detailed math proof about VIO.”
BackgroundMethodInfluentialRD-VIO: Relative-depth-aided visual-inertial odometry for autonomous underwater vehicles
Ding, S., Ma, T., Li, Y., Xu, S., & Yang, Z. (2023) · Applied Ocean Research · doi:10.1016/j.apor.2023.103532 · Cited by 34
Comparison of EKF based SLAM and optimization based SLAM algorithms
Zhang, Y., Zhang, T. L., & Huang, S. (2018) · IEEE Conference on Industrial Electronics and Applications (ICIEA) · doi:10.1109/iciea.2018.8397911 · Cited by 31
“As shown in [18], the NEES at time n is expressed as”
BackgroundLearning Pose Estimation for UAV Autonomous Navigation and Landing Using Visual-Inertial Sensor Data
Baldini, F., Anandkumar, A., & Murray, R. M. (2020) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc45564.2020.9147400 · Cited by 34
Accurate IMU Factor Using Switched Linear Systems for VIO
Henawy, J., Li, Z. G., Yau, W., & Seet, G. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.3000097 · Cited by 33
“The motion integration can be obtained by introducing the IMU kinematic model as [15], [20] ⎧⎨ ⎩ ṖW = VW V̇W = aW ṘBW = RBWω ∧ B (19)”
BackgroundMethodResultInfluentialA Robust Observation, Planning, and Control Pipeline for Autonomous Rendezvous with Tumbling Targets
Albee, K., Oestreich, C., Specht, C., Espinoza, A. T., Todd, J. E., Hokaj, I., Lampariello, R., & Linares, R. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.641338 · Cited by 37
“Since Astrobee’s IMU operates at a much faster rate than the ToF camera, the IMU preintegration theory from (Lupton and Sukkarieh, 2011; Carlone et al., 2014; Forster et al., 2016) is leveraged.”
BackgroundImproved Preintegration Method for GNSS/IMU/In-Vehicle Sensors Navigation Using Graph Optimization
Bai, S.-Y., Lai, J., Lyu, P., Cen, Y., & Ji, B. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3115619 · Cited by 33
Bai, S., Lai, J., Lyu, P., Ji, B., Wang, B., & Sun, X. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3150077 · Cited by 30
“IMU preintegration approach [20], [21] was adapted to derive the error terms associated with asynchronous magnetometer in the IMU/GPS integrated navigation system [22].”
MethodGyro-Net: IMU Gyroscopes Random Errors Compensation Method Based on Deep Learning
Gao, Y., Shi, D., Li, R., Liu, Z., & Sun, W. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3230594 · Cited by 30
“Therefore, there are also some direct or indirect fusion by introducing complementary sensors with IMU, allowing the robotic system to fuse features from multiple sensors and to obtain more robust pose estimation [2].”
BackgroundP 3 -LINS: Tightly Coupled PPP-GNSS/INS/LiDAR Navigation System With Effective Initialization
Li, T., Pei, L., Xiang, Y., Zuo, X., Yu, W., & Truong, T. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3244811 · Cited by 32
Aerial Robots, Visual-Inertial Odometry of
Scaramuzza, D., & Zhang, Z. (2020) · Springer eBooks · doi:10.1007/978-3-642-41610-1_71-1 · Cited by 35
“VIO using the incremental smoothing framework has been demonstrated in Forster et al. (2017a).”
BackgroundMethodSelf-citationLiu, X., Wen, S., Jiang, Z., Tian, W., Qiu, T. Z., & Othman, K. M. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3323964 · Cited by 40
“For IMU preintegration factors, the discrete models of the speed, position, and rotation of the vehicle at time t + (cid:49) t are in [33] Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
MethodBai, S., Wen, W., Li, Y., Shi, C., & Hsu, L. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3470043 · Cited by 35
An Optimization-Based Multi-Sensor Fusion Approach Towards Global Drift-Free Motion Estimation
Wang, K., Cao, C., Ma, S., & Ren, F. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3064446 · Cited by 30
“To avoid that, pre-integration [19] is used.”
An Analytical Solution to the IMU Initialization Problem for Visual-Inertial Systems
Zuñiga-Noël, D., Moreno, F.-Á., & González-Jiménez, J. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3091407 · Cited by 29
Invariant smoothing on Lie Groups
Chauchat, P., Barrau, A., & Bonnabel, S. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594068 · Cited by 32
ARDEA—An MAV with skills for future planetary missions
Lutz, P., Muller, M. G., Maier, M., Stoneman, S., Tomić, T., von Bargen, I., Schuster, M. J., Steidle, F., Wedler, A., Stürzl, W., et al. (2020) · Journal of Field Robotics · doi:10.1002/rob.21949 · Cited by 38
DVI-SLAM: A Dual Visual Inertial SLAM Network
Peng, X., Liu, Z., Li, W., Tan, P., Cho, S. Y., & Wang, Q. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610042 · Cited by 36
“Based on IMU pre-integration [30], the ∆ in Equation 4 and Equation 5, and the pre-integration weight w puij can be estimated.”
MethodHigh-speed aerial grasping using a soft drone with onboard perception
Ubellacker, S., Ray, A., Bern, J. M., Strader, J., & Carlone, L. (2024) · npj Robotics · doi:10.1038/s44182-024-00012-1 · Cited by 31
“…ν k 2 R 6 ∼ N ð 0 ; Σ Þ represents white-noise angular and linear acceleration, and ⊞ maps a vector in R 6 (on the right-hand-side of the operator) to SE (3) (e.g., via an exponential map or, more generally, a retraction 45 ) and then composes it with the pose on the left-hand-side of the operator.”
Self-citationEvent-Based Visual/Inertial Odometry for UAV Indoor Navigation
Elamin, A., El‐Rabbany, A., & Jacob, S. (2024) · Sensors · doi:10.3390/s25010061 · Cited by 38
“The current state is predicted based on the preceding state using the standard IMU kinematics and biases model presented in [51] Then, the IMU error terms are calculated as the difference between the actual state and the predicted state.”
He, D., Li, H., & Yin, J. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3530298 · Cited by 30
Gradient-Based Observer for Simultaneous Localization and Mapping
Zlotnik, D. E., & Forbes, J. R. (2018) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2018.2829467 · Cited by 34
“Further, preintegration on a manifold is considered in [7], where the developed theory is used in conjunction with iSAM2 [8].”
MethodA Real-Time Sliding-Window-Based Visual-Inertial Odometry for MAVs
Xiao, J., Xiong, D., Yu, Q., Huang, K., Lu, H., & Zeng, Z. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2959380 · Cited by 29
“Furthermore, each of them is a linear combination of Gaussian noise; therefore, they are all zero-mean Gaussian distributed [17].”
BackgroundCtrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras
Lang, X., Lv, J., Huang, J., Ma, Y., Liu, Y., & Zuo, X. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3202349 · Cited by 38
LVI-Fusion: A Robust Lidar-Visual-Inertial SLAM Scheme
Liu, Z., Li, Z., Liu, A., Shao, K., Guo, Q., & Wang, C. (2024) · Remote Sensing · doi:10.3390/rs16091524 · Cited by 41
Degradation Resilient LiDAR-Radar-Inertial Odometry
Nissov, M., Khedekar, N., & Alexis, K. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611444 · Cited by 51
Chen, C., Zhu, H. J., Wang, L., & Liu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2961266 · Cited by 31
“To avoid repeatedly calculating the IMU integral after each optimization adjustment, we constrain the relative motion using pre-integration, then parameterized according to the relevant literature [52], [53].”
MethodMultimodal localization: Stereo over LiDAR map
Zuo, X., Ye, W., Yang, Y., Zheng, R., Vidal‐Calleja, T., Huang, G. P., & Liu, Y. (2020) · Journal of Field Robotics · doi:10.1002/rob.21936 · Cited by 27
Li, D., Yang, W., Shi, X., Guo, D., Long, Q., Qiao, F., & Wei, Q. (2020) · IEEE Access · doi:10.1109/access.2020.2989480 · Cited by 27
“The pre-integration [35] is used to calculate the IMU measurement.”
MethodA Novel 3D LiDAR SLAM Based on Directed Geometry Point and Sparse Frame
Liang, S., Cao, Z., Wang, C., & Yu, J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3043200 · Cited by 27
“It is noted that (8) is an optimization problem in manifold SE3, which can be converted to the following issue in Euclidean space by re-parameterization [25]:”
BackgroundCoarse-to-Fine Loosely-Coupled LiDAR-Inertial Odometry for Urban Positioning and Mapping
Zhang, J., Wen, W., Huang, F., Chen, X., & Hsu, L. (2021) · Remote Sensing · doi:10.3390/rs13122371 · Cited by 30
“Please refer to the description from [30] for more detailed derivation.”
BackgroundMethodInfluentialFast and Accurate Initialization for Monocular Vision/INS/GNSS Integrated System on Land Vehicle
Jin, R., Liu, J., Zhang, H., & Niu, X. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3119982 · Cited by 27
“To avoid recom-pute the integration whenever the linearization point changes, we follow [38] and apply the following relative motion increments between two consecutive keyframes that are independent of the pose and velocity at t i : where ω k , a k are angle velocity and acceleration…”
MethodA Tightly Coupled Feature-Based Visual-Inertial Odometry With Stereo Cameras
Yu, L., Qin, J., Wang, S., Wang, Y., & Wang, S. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3176304 · Cited by 33
“α̃ bk bk+1 , β̃k bk+1 , and γ̃ bk bk+1 are the preintegration [27] terms of IMU measurements with noise, which has been discussed in detail in Appendix A of VINS-MONO [25].”
BackgroundMethodThoms, A., Earle, G., Charron, N., & Narasimhan, S. (2023) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2023.3265742 · Cited by 29
“IMU preintegration constraints follow directly from [27], where highfrequency IMU measurements are preintegrated between consecutive keyframes.”
BackgroundInfluentialFAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman FilterPreprint
Xu, W. D., & Zhang, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08196 · Cited by 27
“[15] use a graph optimization with IMU pre-integration constrains [16] and plane constraints [17] from LiDAR feature points.”
MethodDing, S., Zhang, T., Li, Y., Xu, S., & Lei, M. (2024) · Measurement · doi:10.1016/j.measurement.2024.115245 · Cited by 31
Continuous-Time Radar-Inertial and Lidar-Inertial Odometry Using a Gaussian Process Motion Prior
Burnett, K., Schoellig, A. P., & Barfoot, T. D. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3521856 · Cited by 53
“Forster et al. [3] then showed how to perform on-manifold preintegration within the space of 3D rotations, SO (3) .”
BackgroundContinuous-Time State Estimation Methods in Robotics: A Survey
Talbot, W., Nubert, J., Tuna, T., Cadena, C., Dümbgen, F., Tordesillas, J., Barfoot, T. D., & Hutter, M. (2025) · IEEE Transactions on Robotics · doi:10.1109/tro.2025.3593079 · Cited by 59
“…the accumulation of points from a sweeping light detection and ranging (LiDAR) sensor for interscan or scan-to-map registration [4], [5], [6], [7], [8], feature extraction in event [9] or rolling shutter (RS) cameras, and inertial measurement unit (IMU) preintegration [10], [11], [12], [13], [14].”
VinsFusion-Line: Binocular Vision Inertial Navigation Real-Time SLAM System Based on Line Features
Yang, G., Jiang, Y., Wang, S., & Chen, K. (2025) · Cybernetics & Systems · doi:10.1080/01969722.2025.2606002 · Cited by 42
A Visual Positioning System for Indoor Blind Navigation
Zhang, H., & Ye, C. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196782 · Cited by 32
Metadata
- Authors
- Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D.
- Affiliations
- Massachusetts Institute of Technology (US)
- Georgia Institute of Technology (US)
- University of Zurich (CH)
- Venue
- IEEE Transactions on Robotics
- Published
- 2017-02
- Volume
- 33
- Issue
- 1
- Pages
- 1-21
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W2482726005
- Cited by
- 1,502
- References
- 83
- Citations by year
- 2026: 156 · 2025: 179 · 2024: 234 · 2023: 193 · 2022: 180 · 2021: 174 · 2020: 134 · 2019: 129 · 2018: 88 · 2017: 32
- FWCI
- 102.35
- Type
- journal article
- Language
- en
- Open access
- green
- www.zora.uzh.ch
- Licence
- Locations
- IEEE Transactions on Robotics (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Swiss National Foundation (200021-143607)
- National Center of Competence in Research Robotics (NCCR)
- University of Zurich
- National Science Foundation (11115678)
- United States Department of Agriculture (USDA)
- National Institute of Food and Agriculture (GEOW-2014-09160)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Aerospace Engineering › Inertial Sensor and Navigation
- Physical Sciences › Computer Science › Artificial Intelligence › Target Tracking and Data Fusion in Sensor Networks
- Keywords
- visual-inertial odometry, inertial measurement unit, factor graphs, maximum a posteriori estimation, bias correction, real-time state estimation, keyframe selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.