The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Geometry-Aware Learning of Maps for Camera Localization
Abstract
Maps are a key component in image-based camera localization and visual SLAM systems: they are used to establish geometric constraints between images, correct drift in relative pose estimation, and relocalize cameras after lost tracking. The exact definitions of maps, however, are often application-specific and hand-crafted for different scenarios (e.g. 3D landmarks, lines, planes, bags of visual words). We propose to represent maps as a deep neural net called MapNet, which enables learning a data-driven map representation. Unlike prior work on learning maps, MapNet exploits cheap and ubiquitous sensory inputs like visual odometry and GPS in addition to images and fuses them together for camera localization. Geometric constraints expressed by these inputs, which have traditionally been used in bundle adjustment or pose-graph optimization, are formulated as loss terms in MapNet training and also used during inference. In addition to directly improving localization accuracy, this allows us to update the MapNet (i.e., maps) in a self-supervised manner using additional unlabeled video sequences from the scene. We also propose a novel parameterization for camera rotation which is better suited for deep-learning based camera pose regression. Experimental results on both the indoor 7-Scenes dataset and the outdoor Oxford RobotCar dataset show significant performance improvement over prior work. The MapNet project webpage is https://goo.gl/mRB3Au.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346
(2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262033589.001.0001 · Cited by 4,280
Bundle Adjustment — A Modern Synthesis
Triggs, B., McLauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44480-7_21 · Cited by 3,988
LSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · Zaguan (Universidad de Zaragoza) · Cited by 3,153
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.572 · Cited by 2,584
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Modern Multidimensional Scaling: Theory and Applications
Borg, I., & Groenen, P. (2003) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2003.tb01108.x · Cited by 2,197
DTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078
FastSLAM: a factored solution to the simultaneous localization and mapping problem
Montemerlo, M., Thrun, S., Koller, D., & Wegbreit, B. (2002) · doi:10.5555/777092.777184 · Cited by 2,012
NetVLAD: CNN architecture for weakly supervised place recognitionPreprint
Arandjelović, R., Gronát, P., Torii, A., Pajdla, T., & Šivic, J. (2015) · arXiv (Cornell University) · Cited by 1,790
Real-time simultaneous localisation and mapping with a single camera
Davison (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238654 · Cited by 1,733
1 year, 1000 km: The Oxford RobotCar dataset
Maddern, W., Pascoe, G., Linegar, C., & Newman, P. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916679498 · Cited by 1,632
Aggregating Local Image Descriptors into Compact Codes
Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., & Schmid, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.235 · Cited by 1,503
FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472
A Tutorial on Graph-Based SLAM
Grisetti, G., Kummerle, R., Stachniss, C., & Burgard, W. (2010) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2010.939925 · Cited by 1,345
Thrun, S. (2003) · Morgan Kaufmann Publishers Inc. eBooks · Cited by 1,325
Globally Consistent Range Scan Alignment for Environment Mapping
Lu, F., & Milios, E. (1997) · Autonomous Robots · doi:10.1023/a:1008854305733 · Cited by 1,304
Nistér, D., Naroditsky, O., & Bergen, J. R. (2004) · Cited by 1,169
Nister, D., Naroditsky, O., & Bergen, J. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315094 · Cited by 1,030
SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945
Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.377 · Cited by 930
Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing
Dellaert, F., & Kaess, M. (2006) · The International Journal of Robotics Research · doi:10.1177/0278364906072768 · Cited by 922
Geometric Loss Functions for Camera Pose Regression with Deep Learning
Kendall, A., & Cipolla, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.694 · Cited by 803
SBA: A software package for generic sparse bundle adjustment
Lourakis, M. I. A., & Argyros, A. A. (2009) · ACM Transactions on Mathematical Software · doi:10.1145/1486525.1486527 · Cited by 788
Rotations, quaternions, and double groups
Altmann, S. L. (1986) · Cited by 771
Arandjelovic, R., & Zisserman, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.207 · Cited by 713
Metrics for 3D Rotations: Comparison and Analysis
Huynh, D. Q. (2009) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-009-0161-2 · Cited by 698
The Graph SLAM Algorithm with Applications to Large-Scale Mapping of Urban Structures
Thrun, S., & Montemerlo, M. (2006) · The International Journal of Robotics Research · doi:10.1177/0278364906065387 · Cited by 689
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Su, H., Qi, C. R., Li, Y., & Guibas, L. J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.308 · Cited by 680
DSAC — Differentiable RANSAC for Camera Localization
Brachmann, E., Krull, A., Nowozin, S., Shotton, J., Michel, F., Gumhold, S., & Rother, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.267 · Cited by 607
Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization
Sattler, T., Leibe, B., & Kobbelt, L. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2611662 · Cited by 563
PL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments
Gomez-Ojeda, R., Moreno, F.-A., Zuniga-Noel, D., Scaramuzza, D., & Gonzalez-Jimenez, J. (2019) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2899783 · Cited by 560
Modelling uncertainty in deep learning for camera relocalization
Kendall, A., & Cipolla, R. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487679 · Cited by 548 · cited 10×
“Our work is built upon prior works in DNN-based pose estimation methods [11, 30, 31, 32, 40, 55], which regress 6-DoF camera pose from an input RGB image with a DNN.”
BackgroundMethodInfluentialImage-Based Localization Using LSTMs for Structured Feature Correlation
Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.75 · Cited by 521
Semi-dense Visual Odometry for a Monocular Camera
Engel, J., Sturm, J., & Cremers, D. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.183 · Cited by 517
Fast image-based localization using direct 2D-to-3D matching
Sattler, T., Leibe, B., & Kobbelt, L. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126302 · Cited by 480
Strasdat, H., Montiel, J. M. M., & Davison, A. J. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.02.009 · Cited by 439
Modern Multidimensional Scaling
(2005) · Springer Series in Statistics · doi:10.1007/0-387-28981-x · Cited by 432
Worldwide Pose Estimation Using 3D Point Clouds
Li, Y., Snavely, N., Huttenlocher, D., & Fua, P. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_2 · Cited by 366
Robust Rotation and Translation Estimation in Multiview Reconstruction
Martinec, D., & Pajdla, T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383115 · Cited by 319
Quaternions, Interpolation and Animation
Dam, E. B., Koch, M., & Lillholm, M. (2000) · Cited by 300
StructSLAM: Visual SLAM With Building Structure Lines
Zhou, H., Zou, D., Pei, L., Ying, R., Liu, P., & Yu, W. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2015.2388780 · Cited by 280
VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.284 · Cited by 249
Lie-algebraic averaging for globally consistent motion estimation
Govindu, V. M. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315098 · Cited by 245
Point-plane SLAM for hand-held 3D sensors
Taguchi, Y., Jian, Y.-D., Ramalingam, S., & Feng, C. (2013) · 2013 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2013.6631318 · Cited by 214
Real-Time Dense Geometry from a Handheld Camera
Stühmer, J., Gumhold, S., & Cremers, D. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15986-2_2 · Cited by 205
Image-Based Localization Using Hourglass Networks
Melekhov, I., Ylioinas, J., Kannala, J., & Rahtu, E. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.107 · Cited by 197
Combining two-view constraints for motion estimation
Govindu, V. M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990963 · Cited by 196
Towards semantic SLAM using a monocular camera
Civera, J., Galvez-Lopez, D., Riazuelo, L., Tardos, J. D., & Montiel, J. M. M. (2011) · 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2011.6094648 · Cited by 172
Fast, On-Line Learning of Globally Consistent Maps
Duckett, T., Marsland, S., & Shapiro, J. (2002) · Autonomous Robots · doi:10.1023/a:1015269615729 · Cited by 129
Duchenne, O., Audibert, J.-Y., Keriven, R., Ponce, J., & Ségonne, F. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587419 · Cited by 120
Online Photometric Calibration of Auto Exposure Video for Realtime Visual Odometry and SLAM
Bergmann, P., Wang, R., & Cremers, D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2777002 · Cited by 82
Planar Pose Graph Optimization: Duality, Optimal Solutions, and Verification
Carlone, L., Calafiore, G. C., Tommolillo, C., & Dellaert, F. (2016) · IEEE Transactions on Robotics · doi:10.1109/tro.2016.2544304 · Cited by 81
Cited by
From Coarse to Fine: Robust Hierarchical Localization at Large Scale
Sarlin, P.-E., Cadena, C., Siegwart, R., & Dymczyk, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01300 · Cited by 1,044
TartanAir: A Dataset to Push the Limits of Visual SLAM
Wang, W., Zhu, D., Wang, X., Hu, Y., Qiu, Y., Wang, C., Hu, Y., Kapoor, A., & Scherer, S. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341801 · Cited by 456
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression
Sattler, T., Zhou, Q., Pollefeys, M., & Leal-Taixe, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00342 · Cited by 418
L3-Net: Towards Learning Based LiDAR Localization for Autonomous Driving
Lu, W., Zhou, Y., Wan, G., Hou, S., & Song, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00655 · Cited by 311
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
Liang, J., He, R., & Tan, T. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02181-w · Cited by 271
Metadata
- Authors
- Brahmbhatt, S., Gu, J., Kim, K., Hays, J., & Kautz, J.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 2616-2625
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2795645133
- Cited by
- 417
- References
- 57
- FWCI
- 81.68
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.