On-Manifold Preintegration for Real-Time Visual--Inertial Odometry
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Abstract
Current approaches for visual-inertial odometry (VIO) are able to attain highly accurate state estimation via nonlinear optimization. However, real-time optimization quickly becomes infeasible as the trajectory grows over time; this problem is further emphasized by the fact that inertial measurements come at high rate, hence, leading to the fast growth of the number of variables in the optimization. In this paper, we address this issue by preintegrating inertial measurements between selected keyframes into single relative motion constraints. Our first contribution is a preintegration theory that properly addresses the manifold structure of the rotation group. We formally discuss the generative measurement model as well as the nature of the rotation noise and derive the expression for the maximum a posteriori state estimator. Our theoretical development enables the computation of all necessary Jacobians for the optimization and a posteriori bias correction in analytic form. The second contribution is to show that the preintegrated inertial measurement unit model can be seamlessly integrated into a visual-inertial pipeline under the unifying framework of factor graphs. This enables the application of incremental-smoothing algorithms and the use of a structureless model for visual measurements, which avoids optimizing over the 3-D points, further accelerating the computation. We perform an extensive evaluation of our monocular VIO pipeline on real and simulated datasets. The results confirm that our modeling effort leads to an accurate state estimation in real time, outperforming state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 2×
“We use the relative error metrics proposed in [68] to obtain error statistics.”
MethodEstimation with Applications to Tracking and Navigation: Theory, Algorithms and Software
Bar‐Shalom, Y., Li, X. ‐., & Kirubarajan, T. (2002) · doi:10.1002/0471221279 · Cited by 6,782
A Mathematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. S. (2017) · CRC Press eBooks · doi:10.1201/9781315136370 · Cited by 6,694 · cited 1×
“For this purpose, we introduce the following kinematic model [49], [53]:”
MethodFactor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 1×
“This naturally leads to the well known factor graph representation, a class of bipartite graphical models that can be used to represent such factored densities [51], [52].”
BackgroundParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 1×
“A breakthrough has been the development of incremental smoothing techniques (iSAM [36], iSAM2 [3]), which leverage the expressiveness of factor graphs to maintain sparsity and to identify and update only the typically small subset of variables affected by a new measurement.”
MethodMonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · cited 1×
“The complexity of the EKF grows quadratically in the number of estimated landmarks, therefore, a small number of landmarks (in the order of 20) are typically tracked to allow real-time operation [10–12].”
BackgroundMatrix Analysis and Applied Linear Algebra
Meyer, C. (2000) · doi:10.1137/1.9780898719512 · Cited by 3,565 · cited 1×
“Any basis El ∈ R2nl×2nl−3 of the null space of El satisfies the following relation [71]:”
BackgroundMatrix analysis and applied linear algebra
(2001) · Choice Reviews Online · doi:10.5860/choice.38-5065 · Cited by 3,300
Stochastic models, estimation, and control
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6186-7 · Cited by 3,229
Cited by
Items 401–500 of 1,000
Camera-odometer calibration and fusion using graph based optimization
He, Y., Guo, Y., Ye, A., & Yuan, K. (2017) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2017.8324650 · Cited by 21
“Since the EKF methods process a measurement only once, the graph based method do batch optimization allows linearizing multiple times [13].”
MethodSE(2)-Constrained Visual Inertial Fusion for Ground Vehicles
Fan, Z., & Liu, Y. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2873055 · Cited by 21
“Optimization-based methods estimate the state by minimizing a joint nonlinear cost function based on the inertial and visual measurements, and are nowadays considered able to achieve higher accuracy compared to the filtering-based methods, due to that they keep renewing the state linearization point [10].”
BackgroundMethodInfluentialNguyen, T. Q., Mann, G. K. I., Vardy, A., & Gosine, R. G. (2019) · Journal of Dynamic Systems Measurement and Control · doi:10.1115/1.4042951 · Cited by 23
Zhang, Z., Zhao, R., Liu, E., Yan, K., Ma, Y., & Xu, Y. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.03.010 · Cited by 19
SIPs: Succinct Interest Points from Unsupervised Inlierness Probability Learning
Cieslewski, T., Derpanis, K. G., & Scaramuzza, D. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00072 · Cited by 20
“ough an unstructured environment without external help. Examples of state-of-the-art VO/VSLAM systems are [1,2,3,4] or systems that additionally make use of an inertial measurement unit (IMU) such as [5,6,7,8,9]. What most of them have in common is that they extract, as a rst step, a sparse set of interest points from…”
MethodSelf-citationWhite, J. H., & Beard, R. W. (2020) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2020.1003222 · Cited by 19
“Toward that end, and to simplify notation, we define the boxplus operator [14] as”
BackgroundReal-Time Visual-Inertial Localization Using Semantic Segmentation Towards Dynamic Environments
Zhao, X., Wang, C., & Ang, M. H. (2020) · IEEE Access · doi:10.1109/access.2020.3018557 · Cited by 20
“Based on Forster’s pre-integration theory [20] of IMU, Mur-Artal et al. proposed a novel tightly coupled visual-inertial SLAM system ORB-VISLAM [16].”
MethodAdaptive localization and mapping with application to planetary rovers
Hidalgo‐Carrió, J., Poulakis, P., & Kirchner, F. (2018) · Journal of Field Robotics · doi:10.1002/rob.21790 · Cited by 19
“( Forster et al., 2015) and (Forster et al., 2017) propose preintegration of IMU measurements in order to reduce the number of nodes while preserving the manifold structure of the SO (3) rotation group.”
BackgroundZhou, Y., Li, X., Li, S., Wang, X., & Shen, Z. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3393125 · Cited by 22
“For optimization-based implementations, the mainstream methods use a factor graph which jointly optimizes IMU preintegration factors [29] and visual re-projection factors to estimate the navigation states and landmarks [30].”
MethodSchmidt-EKF-based Visual-Inertial Moving Object Tracking
Eckenhoff, K., Geneva, P., Merrill, N., & Huang, G. P. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197352 · Cited by 21
Rotation-Only Bundle Adjustment
Lee, S. H., & Civera, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00049 · Cited by 21
“The two representations are related by Rodrigues formula, and we denote the mapping between them by Exp ( · ) and Log ( · ) [22]: with θ = arccos ((tr( R ) − 1) / 2) .”
Trust Your IMU: Consequences of Ignoring the IMU Drift
Örnhag, M. V., Persson, P., Wadenbäck, M., Åström, K., & Heyden, A. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00493 · Cited by 18
Lyu, Y., Yuan, S., & Xie, L. (2022) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2022.3142663 · Cited by 21
Motion Distortion Elimination for LiDAR-Inertial Odometry Under Rapid Motion Conditions
Shi, J., Wang, W., Li, X., Ye, Y., & Yin, E. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3328695 · Cited by 18
“…dropped and (20) becomes Integrating all IMU data between two LiDAR frames at moments i and j , the attitude preintegration measurement, defined in [28], can be expressed as Inserting (15) on (22) and neglecting the high-order terms, (22) can be rewritten as where b is the gyroscope bias in moment…”
BackgroundMethodInfluentialLIMOT: A Tightly-Coupled System for LiDAR-Inertial Odometry and Multi-Object Tracking
Zhu, Z., Zhao, J., Huang, K., Tian, X., Lin, J., & Ye, C. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3405385 · Cited by 22
“2) IMU Pre-integration: We perform IMU pre-integration to aggregate raw IMU measurements in the local frame, following [15].”
MethodMulti-Radar Inertial Odometry for 3D State Estimation using mmWave Imaging Radar
Huang, J.-T., Xu, R., Hinduja, A., & Kaess, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611194 · Cited by 18
“More details about IMU preintegration on manifold can be found in [23].”
BackgroundMethodField-Testing of a UAV-UGV Team for GNSS-Denied Navigation in Subterranean Environments
Gross, J. N., Petrillo, M. D., Beard, J. J., Nichols, H., Swiger, T., Watson, R. M., Kirk, C., Kiliç, Ç., Hikes, J., Upton, E., et al. (2019) · Proceedings of the Satellite Division's International Technical Meeting (Online)/Proceedings of the Satellite Division's International Technical Meeting (CD-ROM) · doi:10.33012/2019.16912 · Cited by 18
Multi-sensor Fusion Towards VINS: A Concise Tutorial, Survey, Framework and Challenges
Dinh, N. V., & Kim, G.-W. (2020) · IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp48618.2020.00-26 · Cited by 25
“However, it still consumes The full smoothing algorithm [16], [17] employs the whole history states of the travel pathway with their measurement constraints, by solving a large nonlinear optimization with the factor graph algorithm [18].”
MethodLiDAR-Inertial Based Navigation and Mapping for Precision Landing
Setterfield, T. P., Hewitt, R. A., Chen, P., Espinoza, A. T., Trawny, N., & Katake, A. B. (2021) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero50100.2021.9438153 · Cited by 22
“The second term is the the IMU preintegration factor from [29].”
BackgroundMethodInfluentialOn-Orbit Inspection of an Unknown, Tumbling Target using NASA’s Astrobee Robotic Free-Flyers
Oestreich, C., Espinoza, A. T., Todd, J. E., Albee, K., & Linares, R. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00232 · Cited by 20
“Since Astrobee’s IMU operates at a much faster rate than the HazCam, IMU preintegration theory [7] is leveraged to bundle the set of measurements z = { z a τ , z ω τ } = i as a single measurement between keyframes i and j : where ∆ x WC ij represents the odometry of the chaser’s navigational state…”
BackgroundConFusion: Sensor Fusion for Complex Robotic Systems Using Nonlinear Optimization
Sandy, T., Stadelmann, L., Kerscher, S., & Buchli, J. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2894168 · Cited by 23
“Many works have investigated using extended Kalman filters for fiducial marker map building [20], visual odometry [21], [22], and object tracking [18].”
Zhang, Z., Zhao, J., Huang, C., & Li, L. (2022) · Proceedings of the Institution of Mechanical Engineers Part D Journal of Automobile Engineering · doi:10.1177/09544070221102662 · Cited by 19
“where C nb 2 SO(3) represents the rotation matrix from b-frame to n-frame; SO(3) refers to the Special Orthogonal Group, which is a concept to describe the group of 3D rotation matrix in the SLAM and navigation domains.(29)vn, p 2 R(3) and are the velocity and position expressed in n-frame (superscript n); vib is the…”
BackgroundLVIO-SAM: A Multi-sensor Fusion Odometry via Smoothing and Mapping
Zhong, X., Li, Y., Zhu, S., Chen, W., Li, X., & Gu, J. J. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739244 · Cited by 18
“We now can follow [27] to preintegrate the high frequency IMU data between two consecutive lidar frames as shown in Fig.”
MethodObservability-Aware Online Multi-Lidar Extrinsic Calibration
Das, S., af Klinteberg, L., Fallon, M. F., & Chatterjee, S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3262176 · Cited by 21
“The state propagation equations are based on the IMU data [ω : angular velocity, f : linear acceleration] between two timestamps [ti, tj] for all the Δt (inverse of sensor frequency) intervals in the SE(3)manifold is derived in [29] as:”
MethodKang, P., Mu, X., Xu, W., & Xu, Y. (2023) · Journal of Field Robotics · doi:10.1002/rob.22175 · Cited by 19
“This problem can be effectively solved by fusing IMU information (Forster et al., 2017; Qin et al., 2018).”
BackgroundMethodLD-SLAM: A Robust and Accurate GNSS-Aided Multi-Map Method for Long-Distance Visual SLAM
Li, D., Zhang, F., Feng, J., Wang, Z., Fan, J., Li, Y., Li, J., & Yang, T. (2023) · Remote Sensing · doi:10.3390/rs15184442 · Cited by 20
Zhang, Y., Li, B., Sun, S., Liu, Y., Liang, W. G., Xia, X., & Pang, Z. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3317479 · Cited by 18
“To reduce the computational burdens, we seek to preinte-grate the inertial measurements (relative motion increments between t i and t j that are independent of the pose and velocity at t i ) between the successive image frames [41].”
BackgroundNeed for Speed: Fast Correspondence-Free Lidar-Inertial Odometry Using Doppler Velocity
Yoon, D. J., Burnett, K., Laconte, J., Chen, Y., Vhavle, H., Kammel, S., Reuther, J., & Barfoot, T. D. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341596 · Cited by 17
“We do not require pre-integration of the gyroscope data [23], and instead efficiently incorporate data at their exact measurement times.”
MethodA tightly-coupled LIDAR-IMU SLAM method for quadruped robots
Zhou, Z., Zhang, C., Li, C., Zhang, Y., Shi, Y. B., & Zhang, W. (2024) · Measurement and Control · doi:10.1177/00202940231224593 · Cited by 17
Adham, M., Chen, W., Li, Y., & Liu, T. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3487593 · Cited by 25
Continuous Direct Sparse Visual Odometry from RGB-D Images
Jadidi, M. G., Clark, W., Bloch, A. M., Eustice, R. M., & Grizzle, J. W. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.044 · Cited by 26
De-Skewing LiDAR Scan for Refinement of Local Mapping
He, L., Jin, Z. H., & Gao, Z. (2020) · Sensors · doi:10.3390/s20071846 · Cited by 19
“Various optimization approaches are applied on slam, like the graph-based approach [6], Ceres Solver [7], and GTSAM optimization toolbox [8].”
MethodGrid-Based Quaternion Filter for SO(3) Estimation
Li, K., Pfaff, F., & Hanebeck, U. D. (2020) · European Control Conference (ECC) · doi:10.23919/ecc51009.2020.9143723 · Cited by 19
“, the extended Kalman filter (EKF) [6] or unscented Kalman filter (UKF) [7], in a locally linearized space [8]–[10].”
MethodFactor Graph-Based Smoothing Without Matrix Inversion for Highly Precise Localization
Chauchat, P., Barrau, A., & Bonnabel, S. (2020) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2020.3001387 · Cited by 17
“Remark 1: This easily generalizes for smoothing on-manifold (see, for instance, [9], [18], [24]), by simply changing the prior factor and update definitions, and adapting the jacobians accordingly.”
BackgroundMethodInfluentialKFS-LIO: Key-Feature Selection for Lightweight Lidar Inertial Odometry
Li, W., Hu, Y. H., Han, Y., & Li, X. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561324 · Cited by 17
“In addition, the preintegration technique [10] can not only avoid recomputing”
BackgroundMethodInfluentialEgo-Motion Estimation of a Quadrotor Based on Nonlinear Observer
Wang, H., Zheng, D., Wang, J., Chen, W., & Yuan, J. (2018) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2018.2823363 · Cited by 18
“Then, the unscaled velocity is recovered by fusing with other metric measurements [12], [13].”
BackgroundR-LVIO: Resilient LiDAR-Visual-Inertial Odometry for UAVs in GNSS-denied Environment
Zhang, B., Shao, X., Wang, Y., Sun, G., & Yao, W. (2024) · Drones · doi:10.3390/drones8090487 · Cited by 19
Fan, J., Liu, X., Ou, Y., Zhang, P., Zhou, C., & Hou, Z. (2024) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2024.3476931 · Cited by 20
“Detailed derivation and variables explanation of IMU preintegration can be found in [30]. where Z ( x j ) is the depth value of pixel x j , which can be estimated by two-dimensional (2-”
Smartphone-Based Indoor Visual Navigation with Leader-Follower Mode
Xu, J., Dong, E., Ma, Q., Wu, C., & Yang, Z. (2021) · ACM Transactions on Sensor Networks · doi:10.1145/3448417 · Cited by 18
“Others are dedicated to solving the scale uncertainty of monocular SLAM, introducing IMU into monocular visual SLAM [16, 37].”
BackgroundYang, B., Xu, X., Ren, J., Cheng, L., Guo, L., & Zhang, Z. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.11.028 · Cited by 16
Learning Observation Model for Factor Graph Based-State Estimation Using Intrinsic Sensors
Nam, D. V., & Kim, G.-W. (2022) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2022.3193411 · Cited by 16
“Following [39], the noises propagation process is computed given as, where where denotes the right Jacobian [36].”
BackgroundMethodInfluentialHierarchical Distribution-Based Tightly-Coupled LiDAR Inertial Odometry
Wang, C., Cao, Z., Li, J., Yu, J., & Wang, S. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3273288 · Cited by 17
PLI-VIO: Real-time Monocular Visual-inertial Odometry Using Point and Line Interrelated Features
Zhang, J., Yang, J., Shang, Q., & Li, M. (2023) · International Journal of Control Automation and Systems · doi:10.1007/s12555-022-0210-9 · Cited by 16
“Reference [35] reformulated IMU preintegration module in Lie group instead of using Euler angles.”
MethodTightly Coupled LiDAR-Inertial Odometry and Mapping for Underground Environments
Chen, J., Wang, H., & Yang, S. L. (2023) · Sensors · doi:10.3390/s23156834 · Cited by 19
“According to the IMU pre-integration method proposed by [26], since IMU measurements are sampled at a very high rate, we can assume a constant angular velocity and acceleration for the time of integration between two IMU measurements.”
MethodRange-Visual-Inertial Sensor Fusion for Micro Aerial Vehicle Localization and Navigation
Goudar, A., Zhao, W., & Schoellig, A. P. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3335772 · Cited by 16
“Specifically, a generic pose is decomposed into a nominal pose ¯ T i t ∈ SE (3) and a perturbation ξ t ∈ R 6 × 1 : where exp is a retraction operation for SE (3) , ∧ maps an element of R 6 to an element of the associated Lie algebra s e (3) , and ξ t ∼ N ( 0 , Σ i t ) is a Gaussian random variable.”
BackgroundOptimization-Based VINS: Consistency, Marginalization, and FEJ
Chen, C., Geneva, P., Peng, Y., Lee, W., & Huang, G. P. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341637 · Cited by 22
BA-LIOM: tightly coupled laser-inertial odometry and mapping with bundle adjustment
Li, R., Zhang, X., Zhang, S., Yuan, J., Liu, H., & Wu, S. (2024) · Robotica · doi:10.1017/s0263574723001698 · Cited by 16
“[26] is used to calculate the relative body motion between two timesteps.”
MethodTightly-Coupled LiDAR-Visual-Inertial SLAM and Large-Scale Volumetric Occupancy Mapping
Boche, S., Laina, S. B., & Leutenegger, S. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610460 · Cited by 19
“2) IMU Errors: For the formulation of the IMU residuals, OKVIS2 adopts the IMU preintegration approach in [23].”
MethodMulti-visual-inertial system: Analysis, calibration, and estimation
Yang, Y., Geneva, P., & Huang, G. P. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241245726 · Cited by 25
“The new IMU measurement noise n ′ I is computed as: As compared to the conventional IMU pre-integration (Forster et al. 2016) with only mean correction, we have both the mean and covariance corrections when linearization point changes for biases and IMU intrinsics.”
BackgroundMethodInfluentialAn improved SLAM based on RK-VIF: Vision and inertial information fusion via Runge-Kutta method
Cui, J., Zhang, F., Feng, D., Li, C., Li, F., & Tian, Q. (2021) · Defence Technology · doi:10.1016/j.dt.2021.10.009 · Cited by 16
Laser Ranging-Assisted Binocular Visual Sensor Tracking System
Wang, Q., Zhang, Y., Shi, W., & Nie, M. (2020) · Sensors · doi:10.3390/s20030688 · Cited by 16
“Forster C et al. [19] proposed a preintegration theory that aggregates the inertial measurements of adjacent keyframes into independent motion constraints and fuses them into a factor graph framework.”
BackgroundAn Energy-Efficient Quad-Camera Visual System for Autonomous Machines on FPGA Platform
Wan, Z., Zhang, Y., Raychowdhury, A., Bo, Y., Zhang, Y., & Liu, S. (2021) · IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas51828.2021.9458486 · Cited by 16
“A. Visual Frontend Profiling To understand the role of visual frontend, we analyze three popular localization algorithms, SLAM [6], VIO [7] and Registration [8], which are adaptable to different scenarios.”
MethodGR-SLAM: Vision-Based Sensor Fusion SLAM for Ground Robots on Complex Terrain
Su, Y., Wang, T., Yao, C., Shao, S., & Wang, Z. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341387 · Cited by 17
Continuous Integration over SO(3) for IMU Preintegration
Gentil, C. L., & Vidal‐Calleja, T. (2021) · Robotics: Science and Systems · doi:10.15607/rss.2021.xvii.078 · Cited by 18
“The authors of [8, 9] extended the preintegration theory from [15] to perform computations on the SO(3) manifold instead of the original Euler angle representation.”
MethodGrasp synthesis of continuum robots
Mehrkish, A., & Janabi‐Sharifi, F. (2021) · Mechanism and Machine Theory · doi:10.1016/j.mechmachtheory.2021.104575 · Cited by 15
Taghizadeh, S., Nezhadshahbodaghi, M., Safabakhsh, R., & Mosavi, M. R. (2022) · GPS Solutions · doi:10.1007/s10291-022-01265-9 · Cited by 16
Progress in symmetry preserving robot perception and control through geometry and learning
Ghaffari, M., Zhang, R., Zhu, M., Lin, C. E., Lin, T.-Y., Teng, S., Li, T., Liu, T., & Song, J. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.969380 · Cited by 31
“Matrix Lie groups (Chirikjian, 2011; Hall, 2015; Barfoot, 2017) provide natural (exponential) coordinates that exploits symmetries of the space (Long et al., 2013; Barfoot and Furgale, 2014; Forster et al., 2016; Mangelson et al., 2020; Mahony and Trumpf, 2021; Brossard et al., 2022).”
BackgroundAnderson Acceleration for on-Manifold Iterated Error State Kalman Filters
Gao, X., Xiao, T., Bai, C., Zhang, D., & Zhang, F. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3214056 · Cited by 15
“Such on-manifold linearizations are widely used in modern filters and optimizers [20], [21].”
BackgroundLisus, D., Cohen, M. R., & Forbes, J. R. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3268043 · Cited by 16
“2) IMU Preintegration: IMU preintegration, introduced in [30] and extended to MLG theory in [4], provides a way to reduce a large number of IMU measurements into a single relative motion increment (RMI).”
BackgroundMethodAdaptive Fusion Multi-IMU Confidence Level Location Algorithm in the Absence of Stars
Fu, Z., Liu, Y., Si, P., Tao, F., & Wang, N. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3294508 · Cited by 15
“characterization and calibration [4], [5]), measurement integration [6], [7], and sensor fusion [8], [9].”
BackgroundArakawa, R., Zhou, B., Krishnan, G., Goel, M., & Nayar, S. K. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610891 · Cited by 21
Bai, S., Lai, J., Lyu, P., Cen, Y., Sun, X., & Wang, B. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3317275 · Cited by 15
“IMU pre-integration is utilized to offer relative measurements between two successive variable nodes in the graph [42], [43].”
MethodUnderwater Tightly Coupled State Estimation Method Using IMU and Artificial Lateral Line Systems
Ling, Y., Ma, T., Li, Y., Tian, E., Sun, X., Xu, S., & Qi, C. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3381697 · Cited by 16
“In this article, we adopt the definitions of the IMU model and the IMU preintegration residual from [30].”
BackgroundMethodInfluentialPAL-SLAM2: Visual and visual–inertial monocular SLAM for panoramic annular lens
Wang, D., Wang, J., Tian, Y., Fang, Y., Yuan, Z., & Xu, M. (2024) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2024.03.016 · Cited by 15
Wang, Y., Liu, X., Zhao, M., & Xu, X. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13050163 · Cited by 21
“The IMU’s bias can be considered a Wiener process, also known as a random walk or Brownian motion, with its derivative following a Gaussian distribution [25].”
BackgroundMAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration
Wang, Y., Ng, Y., Sa, I., Parra, Á., Rodríguez-Opazo, C., Lin, T., & Li, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10609982 · Cited by 36
A flexible framework for accurate LiDAR odometry, map manipulation, and localization
Blanco, J. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251316881 · Cited by 25
“…we can find how additional sensors have been successfully integrated with LO, leading to LiDAR Inertial odometry (LIO) (Shan et al. 2020; Xu et al. 2022), Visual Inertial Odometry (VIO) (Leutenegger et al. 2015; Forster et al. 2016), or Visual Lidar-Inertial Odometry (VLIO) (Shan et al. 2021).”
Dominguez-Conti, J., Yin, J., Alami, Y., & Civera, J. (2018) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2018.00027 · Cited by 22
“As detailed in [6], this representation is convenient in order to frame the optimization problem in the locally Euclidean tangent space, using a minimal parameterization.”
BackgroundMethodInfluentialOutlier-Robust Manifold Pre-Integration for INS/GPS Fusion
Chng, S.-F., Khosravian, A., Doan, A.-D., & Chin, T.-J. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967643 · Cited by 15
“Although more sophisticated numerical integrated methods can be employed [35]–[38], our experiments suggest that the above Euler approximation performs very well for our specific application where IMU sampling rate is high [39].”
MethodLi, K., Pfaff, F., & Hanebeck, U. D. (2019) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion43075.2019.9011390 · Cited by 16
“Thus, they have been employed for pose representation in various robotic tracking tasks [6], [7].”
MethodHierarchical Multi-Level Information Fusion for Robust and Consistent Visual SLAM
Yu, J., Xiang, Z., & Su, J. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3127554 · Cited by 17
“In addition, the odometer measurement frequency is often higher than that of the camera, so we follow the idea of IMU pre-integration [27] and give the odometer”
BackgroundZhang, Y., Liang, W. G., He, H., & Tan, J. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2795006 · Cited by 15
“[42] proposed a preintegrated IMU model to be seamlessly integrated in a visual-inertial pipeline under the framework of factor graphs on portable google tango.”
MethodVisual–inertial state estimation with camera and camera–IMU calibration
Arbabmir, M., & Ebrahimi, M. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.103249 · Cited by 17
The Complex-Step Derivative Approximation on Matrix Lie Groups
Cossette, C. C., Walsh, A., & Forbes, J. R. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2965882 · Cited by 15
“IMU preintegration as described in [17], but this is beyond the scope of this letter.”
BackgroundAnalytic Combined IMU Integration (ACI2) For Visual Inertial Navigation
Yang, Y., Babu, B. P. W., Chen, C., Huang, G. P., & Ren, L. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197280 · Cited by 27
“Compared to [6], when we compute the IMU integrated covariances, we combine the IMU navigation states and bias states.”
BackgroundMethodInfluentialAn Overview of Perception and Decision-Making in Autonomous Systems in the Era of LearningPreprint
Tang, Y. S., Zhao, C., Wang, J., Zhang, C., Sun, Q., Zheng, W. X., Du, W., Qian, F., & Kurths, J. (2020) · arXiv (Cornell University) · Cited by 16
A Factor-Graph Approach for Optimization Problems with Dynamics ConstraintsPreprint
Xie, M., Escontrela, A., & Dellaert, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06194 · Cited by 25
“They have been successfully applied in other areas of robotics, such as SLAM [35], [34], [14], [20] and state estimation in legged robots [28], [64].”
MethodSelf-citationTeng, X., Xu, P., Guo, D., Guo, Y., Hu, R., Chai, H., & Chuxing, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341684 · Cited by 14
“Therefore, they suffer position drifts since errors accumulate rapidly in the integration process [11], [12].”
BackgroundMaximum A Posteriori estimation for AUV localization with USBL measurements
Franchi, M., Bucci, A., Zacchini, L., Ridolfi, A., Bresciani, M., Peralta, G., & Costanzi, R. (2021) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.10.109 · Cited by 16
Monocular Visual-Inertial and Robotic-Arm Calibration in a Unifying Framework
Zhang, Y., Liang, W. G., Yuan, M., He, H., Tan, J., & Pang, Z. (2021) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2021.1004290 · Cited by 15
“Camera and IMU share heterogeneous features regarding their quick response, tracking accuracy, and absolute pose observability [6]–[9].”
BackgroundCORB2I-SLAM: An Adaptive Collaborative Visual-Inertial SLAM for Multiple Robots
Saha, A., Dhara, B. C., Umer, S., AlZubi, A. A., Alanazi, J. M., & Kulakov, Y. (2022) · Electronics · doi:10.3390/electronics11182814 · Cited by 14
Visual Heading-Aided Pedestrian Navigation Method Based on Factor Graph in Indoor Environment
Yuan, C., Lai, J., Lyu, P., Wang, T., & Zhu, J. (2023) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2023.3250749 · Cited by 14
“[18], the integration coordinate is converted from the world frame, which is absolute to the relative body frame.”
BackgroundVersatile LiDAR-Inertial Odometry With SE(2) Constraints for Ground Vehicles
Chen, J., Wang, H., Hu, M., & Suganthan, P. N. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3268584 · Cited by 17
“The aim of the system is to determine the state of the robot based on the measurements gathered from the LiDAR and IMU sensors, which is a Maximum A Posteriori (MAP) estimation problem [26].”
BackgroundMethodLocalization and Motion Planning of Industrial Tractor–Trailers Vehicles
Zhao, H., Chen, W., Zhou, S., Fan, Z., & Liu, Y. (2023) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2023.3275497 · Cited by 15
“Given the prior configuration X 0 and all the probabilistic measurements Z , the goal is to find the optimal MAP estimate X ⋆ , which corresponds to the maximum value of the following equation : Equivalently, X ⋆ is also associated with the minimum of the negative log-posterior of (5) [13].”
BackgroundContinuous latent state preintegration for inertial-aided systems
Gentil, C. L., & Vidal‐Calleja, T. (2023) · The International Journal of Robotics Research · doi:10.1177/02783649231199537 · Cited by 14
Delayed Kalman filter for vision-based autonomous flight in ocean environments
Gamagedara, K., Lee, T., & Snyder, M. R. (2023) · Control Engineering Practice · doi:10.1016/j.conengprac.2023.105791 · Cited by 14
Visual-Inertial-Laser-Lidar (VILL) SLAM: Real-Time Dense RGB-D Mapping for Pipe Environments
Tian, T., Wang, L., Yan, X., Ruan, F., Aadityaa, G. J., Choset, H., & Li, L. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341761 · Cited by 21
“After acquiring data from each physical sensor, a series of front-end preprocessing processes formulate data frames by tracking the visual features, preintegrating IMU measurements [17], triangulating the laser profiler data, and fitting cylinder primitives using the Lidar point cloud.”
MethodIPR-VINS: Real-time monocular visual-inertial SLAM with implicit plane optimization
Zhang, D., Shen, Y., Lu, J., Jiang, Q., Zhao, C. K., & Yang, M. (2024) · Measurement · doi:10.1016/j.measurement.2023.114099 · Cited by 14
Real-Time Cross-View Image Matching and Camera Pose Determination for Unmanned Aerial Vehicles
Chen, L., Wu, B., Duan, R., & Chen, Z. (2024) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.23-00073r2 · Cited by 16
An improved Multi-State Constraint Kalman Filter for Visual-Inertial Odometry
Abdollahi, M. R., Pourtakdoust, S. H., Yoosefian, M. H., & Pishkenari, H. N. (2024) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2024.107130 · Cited by 17
“To have a more general comparison, the results of SVO (VIO) [27], SVO+GTSAM [3], and ROVIO [9] algorithms are also presented.”
MethodResultInfluentialCoLRIO: LiDAR-Ranging-Inertial Centralized State Estimation for Robotic Swarms
Zhong, S., Chen, H., Qi, Y., Feng, D., Chen, Z., Wu, J., Wen, W., & Liu, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611672 · Cited by 16
“With a time offset D t between two frames, the IMU preintegration factor r I can be expressed as: where (cid:136) R , (cid:136) p , and (cid:136) v indicates the estimated rotation, translation, and velocity.”
BackgroundOkawara, T., Koide, K., Oishi, S., Yokozuka, M., Banno, A., Uno, K., & Yoshida, K. (2024) · IEEE Access · doi:10.1109/access.2024.3461655 · Cited by 14
“The error e IMU ij between x i and x j is finally defined as follows: Further details can be found in [18].”
BackgroundSong, J., Bagoren, O., Andigani, R., Sethuraman, A. V., & Skinner, K. A. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801692 · Cited by 32
“Our proposed DVL factor residual is formulated as The Jacobians for the involved state variables p i and p j can be derived following [16].”
InfluentialGravityFusion: Real-time dense mapping without pose graph using deformation and orientation
Puri, P., Jia, D., & Kaess, M. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206559 · Cited by 14
“[3] improve upon the preintegration method used for integrating the inertial data into the SLAM factor graph.”
MethodAbsolute humanoid localization and mapping based on IMU Lie group and fiducial markers
Fourmy, M., Atchuthan, D., Mansard, N., Solà, J., & Flayols, T. (2019) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids43949.2019.9035005 · Cited by 26
“Mathematically, and disregarding methodology, the main difference between our method and Forster’s [12] is to be found in the exponential map.”
BackgroundMethodInfluentialPerception-aided Visual-Inertial Integrated Positioning in Dynamic Urban Areas
Bai, X., Zhang, B., Wen, W., Hsu, L., & Li, H. (2020) · IEEE/ION Position Location and Navigation Symposium · doi:10.1109/plans46316.2020.9109963 · Cited by 16
“We employ the state-of-the-art IMU pre-integration technique [36, 37] to integrate the IMU measurements, which can effectively alleviate the high computation load in FGO and the accuracy is guaranteed, by integrating multiple IMU measurements into a single factor in FGO.”
MethodImproved Point–Line Visual–Inertial Odometry System Using Helmert Variance Component Estimation
Xu, B., Chen, Y., Zhang, S., & Wang, J. (2020) · Remote Sensing · doi:10.3390/rs12182901 · Cited by 16
“The front end is designed for the processing of visual measurement information, the preintegration of IMU measurement information [8], and calculates the initial poses.”
MethodA Robust Optical Flow Tracking Method Based On Prediction Model for Visual-Inertial Odometry
Zhu, S., Yang, S., Hu, P., & Qu, X. H. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3079806 · Cited by 16
“This process follows the quaternion-based derivation of IMU integration in continuous time [28] and handling of IMU biases [29].”
MethodSupSLAM: A Robust Visual Inertial SLAM System Using SuperPoint for Unmanned Aerial Vehicles
Quach, C. H., Phung, M. D., Le, H. V., & Perry, S. W. (2021) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics54270.2021.9701527 · Cited by 20
A pedestrian POS for indoor Mobile Mapping System based on foot-mounted visual–inertial sensors
Niu, X., Yan, W., & Kuang, J. (2022) · Measurement · doi:10.1016/j.measurement.2022.111559 · Cited by 17
“Thus, the cost function using the IMU measurement can be written as: here, 𝑒 𝑝𝑟𝑒 − 𝑖𝑛𝑡𝑒𝑔𝑟𝑎𝑡𝑖𝑜𝑛 denote the cost function of pre-integration on manifold [32], 𝒗 𝑧𝑒𝑟𝑜 = [0 , 0 , 0] 𝑇 meant the zero velocity vector.”
Metadata
- Authors
- Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D.
- Affiliations
- Massachusetts Institute of Technology (US)
- Georgia Institute of Technology (US)
- University of Zurich (CH)
- Venue
- IEEE Transactions on Robotics
- Published
- 2017-02
- Volume
- 33
- Issue
- 1
- Pages
- 1-21
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W2482726005
- Cited by
- 1,502
- References
- 83
- Citations by year
- 2026: 156 · 2025: 179 · 2024: 234 · 2023: 193 · 2022: 180 · 2021: 174 · 2020: 134 · 2019: 129 · 2018: 88 · 2017: 32
- FWCI
- 102.35
- Type
- journal article
- Language
- en
- Open access
- green
- www.zora.uzh.ch
- Licence
- Locations
- IEEE Transactions on Robotics (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Swiss National Foundation (200021-143607)
- National Center of Competence in Research Robotics (NCCR)
- University of Zurich
- National Science Foundation (11115678)
- United States Department of Agriculture (USDA)
- National Institute of Food and Agriculture (GEOW-2014-09160)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Aerospace Engineering › Inertial Sensor and Navigation
- Physical Sciences › Computer Science › Artificial Intelligence › Target Tracking and Data Fusion in Sensor Networks
- Keywords
- visual-inertial odometry, inertial measurement unit, factor graphs, maximum a posteriori estimation, bias correction, real-time state estimation, keyframe selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.