The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Modelling uncertainty in deep learning for camera relocalization
Open accessAbstract
We present a robust and real-time monocular six degree of freedom visual relocalization system. We use a Bayesian convolutional neural network to regress the 6-DOF camera pose from a single RGB image. It is trained in an end-to-end manner with no need of additional engineering or graph optimisation. The algorithm can operate indoors and outdoors in real time, taking under 6ms to compute. It obtains approximately 2m and 6° accuracy for very large scale outdoor scenes and 0.5m and 10° accuracy indoors. Using a Bayesian convolutional neural network implementation we obtain an estimate of the model's relocalization uncertainty and improve state of the art localization accuracy on a large scale outdoor dataset. We leverage the uncertainty measure to estimate metric relocalization error and to detect the presence or absence of the scene in the input image. We show that the model's uncertainty is caused by images being dissimilar to the training dataset in either pose or appearance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We adopted the ResNet architectures with pre-trained weights as the network backbone, due to its low number of pooling layers, and good accuracy-complexity trade-off [35].”
From Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering (2020) · cites this paper as [35]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Convolutional neural networks have recently been shown to be extremely powerful for generating image representations for classification [12], [13].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 3×
“This has been achieved using SIFT [2] features with FAB-MAP [6] or using convolutional neural network features [7].”
BackgroundMethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 4×
“Additionally, in green we show the output from the first auxiliary pose regressor from the GoogLeNet architecture (see Figure 3 of [13]).”
BackgroundMethodInfluential(2014) · Cited by 33,419
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“These higher level contours and shape cues [14] have been shown to be increasingly invariant to object instantiation parameters.”
BackgroundCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1×
“Using standard libraries such as [25] we can obtain an efficient implementation of a Bernoulli Bayesian convolutional neural network.”
MethodORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037 · cited 2×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundMethodThrun, S. (2002) · CACM · doi:10.1145/504729.504754 · Cited by 7,941 · cited 1×
“nsider localization uncertainty [1], [5], [11], previous probabilistic algorithms have been proposed. Bayesian approaches include extended Kalman filters and particle filter approaches such as FastSLAM [18]. However these approaches estimate uncertainty from sensor noise models, not the uncertainty of the model to repre…”
MethodORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · cited 1×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundDropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · cited 1×
“Recent work [23] has extended dropout to approximate Bayesian inference over the network’s weights.”
BackgroundHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“[28] argues that transferring weights can cause performance to drop in two situations.”
BackgroundCaffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 5,299 · cited 1×
“Using standard libraries such as [25] we can obtain an efficient implementation of a Bernoulli Bayesian convolutional neural network.”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 8×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundInfluentialLSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · cited 3×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundDeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.214 · Cited by 3,383 · cited 1×
“However their use for regression has been predominantly limited to object detection [13] and human pose regression [15].”
BackgroundA practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · cited 1×
“Neural networks which consider uncertainty are known as Bayesian neural networks [19], [20].”
Background(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Self-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 2×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundVariational inference for Dirichlet process mixtures
Blei, D. M., & Jordan, M. I. (2006) · Bayesian Analysis · doi:10.1214/06-ba104 · Cited by 1,491 · cited 1×
“However we found that all of these methods produced poorer localization uncertainty than fitting a single unimodal Gaussian to the data.”
MethodFAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472 · cited 1×
“This has been achieved using SIFT [2] features with FAB-MAP [6] or using convolutional neural network features [7].”
MethodPractical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079
Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A. W. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.377 · Cited by 952 · cited 7×
“A learning approach to relocalization has also been proposed previously using random forests to regress Scene Coordinate labels [16].”
MethodInfluentialBayesian Convolutional Neural Networks with Bernoulli Approximate Variational InferencePreprint
Gal, Y., & Ghahramani, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02158 · Cited by 777 · cited 7×
“The results in Table 6 show that using Monte Carlo dropout [24] results in a considerable improvement in localization accuracy over PoseNet [8].”
BackgroundMethodResultInfluentialPlace Recognition with ConvNet Landmarks: Viewpoint-Robust, Condition-Robust, Training-Free
Sünderhauf, N., Rowlands, S., Jacobson, A., Dayoub, F., Pepperell, E., Upcroft, B., & Milford, M. J. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.022 · Cited by 374
Worldwide Pose Estimation Using 3D Point Clouds
Li, Y., Snavely, N., Huttenlocher, D. P., & Fua, P. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_2 · Cited by 369 · cited 3×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundTransforming Neural-Net Output Levels to Probability Distributions
Denker, J. S., & LeCun, Y. (1990) · Cited by 223
Exploiting uncertainty in regression forests for accurate camera relocalization
Valentin, J., Niebner, M., Shotton, J., Fitzgibbon, A., Izadi, S., & Torr, P. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299069 · Cited by 156 · cited 1×
“It was extended with a probabilistic approach in [17].”
Coarse-to-fine vision-based localization by indexing scale-Invariant features
Wang, J., Zha, H., & Cipolla, R. (2006) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859085 · Cited by 136 · cited 1×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundSelf-citationConvolutional networks for real-time 6-DOF camera relocalization.Preprint
Kendall, A., Grimes, M. K., & Cipolla, R. (2015) · arXiv (Cornell University) · Cited by 50
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
“uncertainty quantity technique [397], [43], [249], [420], [394], [413], [10], [411], [418], applicable on various deep learning architectures.”
BackgroundInfluentialPoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489
Self-citationKendall, A., Badrinarayanan, V., & Cipolla, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.57 · Cited by 784
“This technique has seen success in modelling uncertainty for camera relocalisation [19].”
MethodSelf-citationGeometric Loss Functions for Camera Pose Regression with Deep Learning
Kendall, A., & Cipolla, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.694 · Cited by 803
“Later work has extended PoseNet to use RGB-D input [25], learn relative ego-motion [31], improve the context of features [54], localise over video sequences [8] and interpret relocalisation uncertainty with Bayesian Neural Networks [19].”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Kendall, A., & Cipolla, R.
- Affiliations
- University of Cambridge (GB)
- ORCID
- Cipolla, Roberto
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 4762-4769
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2279895976
- Cited by
- 548
- References
- 30
- Citations by year
- 2026: 20 · 2025: 32 · 2024: 44 · 2023: 57 · 2022: 52 · 2021: 68 · 2020: 90 · 2019: 82 · 2018: 57 · 2017: 39 · 2016: 6
- FWCI
- 107.26
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- camera relocalization, Bayesian convolutional neural network, uncertainty estimation, 6D pose estimation, real-time localization, out-of-distribution detection, indoor and outdoor scenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
