The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry
Open accessAbstract
In this tutorial, we provide principled methods to quantitatively evaluate the quality of an estimated trajectory from visual(-inertial) odometry (VO/VIO), which is the foundation of benchmarking the accuracy of different algorithms. First, we show how to determine the transformation type to use in trajectory alignment based on the specific sensing modality (i.e., monocular, stereo and visual-inertial). Second, we describe commonly used error metrics (i.e., the absolute trajectory error and the relative error) and their strengths and weaknesses. To make the methodology presented for VO/VIO applicable to other setups, we also generalize our formulation to any given sensing modality. To facilitate the reproducibility of related research, we publicly release our implementation of the methods described in this tutorial.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · cited 1×
“ATE is also widely used to evaluate visual odometry/SLAM algorithms, for example, in [2], [3], [4].”
MethodVINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
Qin, T., Li, P., & Shen, S. (2018) · IEEE Transactions on Robotics · doi:10.1109/tro.2018.2853729 · Cited by 4,669 · cited 2×
“We can see the “raw” estimation from VINS-Mono is in a different reference frame as the groundtruth, and therefore cannot be directly compared.”
MethodA benchmark for the evaluation of RGB-D SLAM systems
Sturm, J., Engelhard, N., Endres, F., Burgard, W., & Cremers, D. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6385773 · Cited by 3,976
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153
Engel, J., Koltun, V., & Cremers, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,924
Least-squares estimation of transformation parameters between two point patterns
Umeyama, S. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.88573 · Cited by 2,420
The EuRoC micro aerial vehicle datasets
Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M. W., & Siegwart, R. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915620033 · Cited by 2,066
On-Manifold Preintegration for Real-Time Visual--Inertial Odometry
Forster, C., Carlone, L., Dellaert, F., & Scaramuzza, D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2597321 · Cited by 1,502 · cited 3×
“In this paper, we use the preintegrated IMU measurements proposed in [9], [10].”
BackgroundMethodSelf-citationSVO: Semidirect Visual Odometry for Monocular and Multicamera Systems
Forster, C., Zhang, Z., Gassner, M., Werlberger, M., & Scaramuzza, D. (2017) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2623335 · Cited by 922
Self-citationVisual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration
Kelly, J., & Sukhatme, G. S. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364910382802 · Cited by 536
Lupton, T., & Sukkarieh, S. (2012) · IEEE Transactions on Robotics · doi:10.1109/tro.2011.2170332 · Cited by 469
On measuring the accuracy of SLAM algorithms
Kümmerle, R., Steder, B., Dornhege, C., Ruhnke, M., Grisetti, G., Stachniss, C., & Kleiner, A. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9155-6 · Cited by 331
Batch Continuous-Time Trajectory Estimation as Exactly Sparse Gaussian Process Regression
Barfoot, T., Tong, C. H., & Särkkä, S. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.001 · Cited by 164
A comparison of SLAM algorithms based on a graph of relations
Burgard, W., Stachniss, C., Grisetti, G., Steder, B., Kummerle, R., Dornhege, C., Ruhnke, M., Kleiner, A., & Tardós, J. D. (2009) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2009.5354691 · Cited by 128
On the Comparison of Gauge Freedom Handling in Optimization-Based Visual-Inertial State Estimation
Zhang, Z., Gallego, G., & Scaramuzza, D. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2833152 · Cited by 27
Self-citation
Cited by
D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry
Yang, N., von Stumberg, L., Wang, R., & Cremers, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00136 · Cited by 438
Lin, J., & Zhang, F. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9811935 · Cited by 423
Events-To-Video: Bringing Modern Computer Vision to Event Cameras
Rebecq, H., Ranftl, R., Koltun, V., & Scaramuzza, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00398 · Cited by 420
Self-citationVoxelized GICP for Fast and Accurate 3D Point Cloud Registration
Koide, K., Yokozuka, M., Oishi, S., & Banno, A. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560835 · Cited by 386
R $^2$ LIVE: A Robust, Real-Time, LiDAR-Inertial-Visual Tightly-Coupled State Estimator and Mapping
Lin, J., Zheng, C., Xu, W., & Zhang, F. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3095515 · Cited by 293
Metadata
- Authors
- Zhang, Z., & Scaramuzza, D.
- Venue
- 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2018-10
- Pages
- 7244-7251
- OpenAlex
- W2909157769
- Cited by
- 658
- References
- 16
- FWCI
- 86.95
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.