The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Long-Term Visual Localization Revisited
Abstract
Visual localization enables autonomous vehicles to navigate in their surroundings and augmented reality applications to link virtual to real worlds. Practical visual localization approaches need to be robust to a wide variety of viewing conditions, including day-night changes, as well as weather and seasonal variations, while providing highly accurate six degree-of-freedom (6DOF) camera pose estimates. In this paper, we extend three publicly available datasets containing images captured under a wide variety of viewing conditions, but lacking camera pose information, with ground truth pose information, making evaluation of the impact of various factors on 6DOF camera pose estimation accuracy possible. We also discuss the performance of state-of-the-art localization approaches on these datasets. Additionally, we release around half of the poses for all conditions, and keep the remaining half private as a test set, in the hopes that this will stimulate research on long-term visual localization, learned local image features, and related research areas. Our datasets are available at visuallocalization.net, where we are also hosting a benchmarking server for automatic evaluation of results on the test set. The presented state-of-the-art results are to a large degree based on submissions to our server.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“This can be done using e.g. Cityscapes classes [20], or more fine-grained semantic classes learned in a self-supervised manner [41].”
MethodVision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Video Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · cited 2×
“…where robustness to changing conditions is critical for long-term autonomous navigation [17], [47], [57], [60], [83], [89]: FAB-MAP [21] is an image retrieval approach based on the Bag-of-Words (BoW) paradigm [78] that explicitly models the co-occurrence probability of different visual words.”
MethodPoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
1 year, 1000 km: The Oxford RobotCar dataset
Maddern, W., Pascoe, G., Linegar, C., & Newman, P. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916679498 · Cited by 1,632 · cited 2×
“Our RobotCar Seasons dataset is based on a subset of the publicly available Oxford RobotCar Dataset [54].”
BackgroundMethodSelf-citationFAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472 · cited 3×
“Landmarks for Place Recognition [95] 0.0 / 0.2 / 20.8 0.0 / 1.0 / 10.2 FAB-MAP [21] 0.0 / 0.0 / 4.6 0.0 / 0.0 / 0.0 It is interesting to note that while the hierarchical methods experience a large drop for night-time images in the high-precision regime, the drop in accuracy for the coarse-precision…”
BackgroundMethodInfluentialVisual Place Recognition: A Survey
Lowry, S., Sünderhauf, N., Newman, P., Leonard, J. J., Cox, D., Corke, P., & Milford, M. J. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2496823 · Cited by 1,079
SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights
Milford, M. J., & Wyeth, G. F. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6224623 · Cited by 981 · cited 5×
“Sequence-based approaches for image retrieval are used for loop-closure detection in robotics [53], [57], [61].”
BackgroundMethodResultInfluentialBenchmarking 6DOF Outdoor Visual Localization in Changing Conditions
Sattler, T., Maddern, W., Toft, C., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00897 · Cited by 699 · cited 4×
“Two of the datasets presented here are extended versions of the datasets in [70]: the Extended CMU Seasons contains roughly 40% more images than the original CMU Seasons dataset (mostly of challenging vegetated areas).”
BackgroundMethodInfluentialSelf-citationUniversity of Michigan North Campus long-term vision and lidar dataset
Carlevaris-Bianco, N., Ushani, A. K., & Eustice, R. M. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915614638 · Cited by 576
Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image
Brachmann, E., Michel, F., Krull, A., Yang, M. Y., Gumhold, S., & Rother, C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.366 · Cited by 530
Image-Based Localization Using LSTMs for Structured Feature Correlation
Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.75 · Cited by 521 · cited 2×
“They learn features with stable appearance over time [17], [22], [24], [58], [60], [64], [74], [91], train classifiers for place recognition [13], [31], [36], [39], [47], [94], and train CNNs to regress 2D-3D matches [9], [10], [76] or camera poses [19], [37], [92].”
BackgroundMethodSelf-citationVidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.284 · Cited by 249
Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization
Lynen, S., Sattler, T., Bosse, M., Hesch, J., Pollefeys, M., & Siegwart, R. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.037 · Cited by 243 · cited 2×
“Descriptor matching can be accelerated by prioritization [18], [45], [69] and efficient search algorithms [23], [52].”
BackgroundSelf-citationTorontoCity: Seeing the World with a Million Eyes
Wang, S., Bai, M., Mattyus, G., Chu, H., Luo, W., Yang, B., Liang, J., Cheverie, J., Fidler, S., & Urtasun, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.327 · Cited by 186 · cited 1×
“Datasets such as KITTI [29], TorontoCity [93], or the M´alaga Urban dataset [7] also provide street-level image sequences.”
BackgroundA Cross-Season Correspondence Dataset for Robust Semantic Segmentation
Mans Larsson, M., Stenborg, E., Hammarstrand, L., Pollefeys, M., Sattler, T., & Kahl, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00976 · Cited by 71
Self-citation
Cited by
LoFTR: Detector-Free Local Feature Matching with Transformers
Sun, J., Shen, Z., Wang, Y., Bao, H., & Zhou, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00881 · Cited by 1,719
Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition
Hausler, S., Garg, S., Xu, M., Milford, M., & Fischer, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01392 · Cited by 444
Back to the Feature: Learning Robust Camera Localization from Pixels to Pose
Sarlin, P.-E., Unagar, A., Larsson, M., Germain, H., Toft, C., Larsson, V., Pollefeys, M., Lepetit, V., Hammarstrand, L., Kahl, F., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00326 · Cited by 297
Self-citationPauly, L., Rharbaoui, W., Shneider, C., Rathinam, A., Gaudillière, V., & Aouada, D. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.08.001 · Cited by 121
“More subtly, a domain gap persists even when the real source and target datasets are acquired under different (laboratory and space) environmental conditions [165].”
BackgroundIs Geometry Enough for Matching in Visual Localization?
Zhou, Q., Agostinho, S., Osep, A., & Leal-Taixé, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12979 · Cited by 52
“Structure-based localization methods achieve impressive localization accuracy and state-of-the-art performance [47, 16, 48] in the longterm localization benchmark [51, 64], containing extreme appearance changes and a high viewpoint variation.”
Background
Metadata
- Authors
- Toft, C., Maddern, W., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., Pajdla, T., Kahl, F., & Sattler, T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2022-04-01
- Volume
- 44
- Issue
- 4
- Pages
- 2074-2088
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3093386634
- Cited by
- 190
- References
- 21
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.