The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Modelling uncertainty in deep learning for camera relocalization
Open accessAbstract
We present a robust and real-time monocular six degree of freedom visual relocalization system. We use a Bayesian convolutional neural network to regress the 6-DOF camera pose from a single RGB image. It is trained in an end-to-end manner with no need of additional engineering or graph optimisation. The algorithm can operate indoors and outdoors in real time, taking under 6ms to compute. It obtains approximately 2m and 6° accuracy for very large scale outdoor scenes and 0.5m and 10° accuracy indoors. Using a Bayesian convolutional neural network implementation we obtain an estimate of the model's relocalization uncertainty and improve state of the art localization accuracy on a large scale outdoor dataset. We leverage the uncertainty measure to estimate metric relocalization error and to detect the presence or absence of the scene in the input image. We show that the model's uncertainty is caused by images being dissimilar to the training dataset in either pose or appearance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We adopted the ResNet architectures with pre-trained weights as the network backbone, due to its low number of pooling layers, and good accuracy-complexity trade-off [35].”
From Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering (2020) · cites this paper as [35]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“Convolutional neural networks have recently been shown to be extremely powerful for generating image representations for classification [12], [13].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 3×
“This has been achieved using SIFT [2] features with FAB-MAP [6] or using convolutional neural network features [7].”
BackgroundMethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 4×
“Additionally, in green we show the output from the first auxiliary pose regressor from the GoogLeNet architecture (see Figure 3 of [13]).”
BackgroundMethodInfluential(2014) · Cited by 33,419
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“These higher level contours and shape cues [14] have been shown to be increasingly invariant to object instantiation parameters.”
BackgroundCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1×
“Using standard libraries such as [25] we can obtain an efficient implementation of a Bernoulli Bayesian convolutional neural network.”
MethodORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037 · cited 2×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundMethodThrun, S. (2002) · CACM · doi:10.1145/504729.504754 · Cited by 7,941 · cited 1×
“nsider localization uncertainty [1], [5], [11], previous probabilistic algorithms have been proposed. Bayesian approaches include extended Kalman filters and particle filter approaches such as FastSLAM [18]. However these approaches estimate uncertainty from sensor noise models, not the uncertainty of the model to repre…”
MethodORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · cited 1×
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundDropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · cited 1×
“Recent work [23] has extended dropout to approximate Bayesian inference over the network’s weights.”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–200 of 549
Considerations for a more ethical approach to data in AI: On data representation and infrastructure
Baird, A., & Schuller, B. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00025 · Cited by 46
“Given the “black-box” nature of deep learning, there have been numerous approaches to quantifying confidence (Kendall and Cipolla, 2016; Keren et al., 2018).”
BackgroundA Review of Recurrent Neural Network Based Camera Localization for Indoor Environments
Alam, M. S., Mohamed, F., Selamat, A., & Hossain, A. B. (2023) · IEEE Access · doi:10.1109/access.2023.3272479 · Cited by 46
“The real benefit of Bayesian-PoseNet [56] is that it can be extended to a Bayesian model that can estimate localization uncertainty.”
BackgroundMethodInfluentialMap-Relative Pose Regression for Visual Re-Localization
Chen, S., Cavallari, T., Prisacariu, V. A., & Brachmann, E. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01953 · Cited by 43
“Kendall et al. introduced the first APR approach, named PoseNet [23– 25], where a feed-forward neural network directly regresses a 7-dimensional pose vector for every query image.”
BackgroundCavallari, T., Bertinetto, L., Mukhoti, J., Torr, P. H. S., & Golodetz, S. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00068 · Cited by 47
“For Cambridge Landmarks [36, 34, 35], the training sequences contain many moving objects (pedestrians, cars, etc.).”
BackgroundMethodInfluentialSimVODIS: Simultaneous Visual Odometry, Object Detection, and Instance Segmentation
Kim, U.-H., Kim, S.-H., & Kim, J.-H. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3007546 · Cited by 43
Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision
Türkoglu, M. Ö., Brachmann, E., Konrad Schindler, B., Brostow, G., & Monszpart, Á. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00025 · Cited by 45
“Bayes-PoseNet [34] uses a Bayesian CNN to also obtain an estimate of the model’s re-localization uncertainty.”
MethodWang, K., Wang, Y., Liu, B., & Chen, J. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3256459 · Cited by 49
“1) Visual Localization: In the study of visual uncertainty, Kendall and Cipolla [92] proposed a robust and real-time monocular 6-degree of freedom (DOF) visual repositioning system.”
BackgroundImage and Object Geo-Localization
Wilson, D., Zhang, X., Sultani, W., & Wshah, S. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01942-3 · Cited by 47
“Kendall and Cipolla (2016) proposed a CNN architecture and Bayesian network capable of regressing the 6-DOF (degree of freedom) camera poses of RGB images.”
Localising Faster: Efficient and precise lidar-based robot localisation in large-scale environments
Sun, L., Adolfsson, D., Magnusson, M., Andreasson, H., Posner, I., & Duckett, T. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196708 · Cited by 42
“Deep learning methods [8], [9], [10], [11], [12] are emerging in imagebased relocalisation as a pivotal precursor to directly estimate the 6-DOF pose based on a model learned for a specific environment with a deep regression neural network.”
BackgroundMethodImage-based Localization with Spatial LSTMs.Preprint
Walch, F., Hazırbaş, C., Leal-Taixé, L., Sattler, T., Hilsenbeck, S., & Cremers, D. (2016) · arXiv (Cornell University) · Cited by 43
The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
Zhao, X., Agrawal, H., Batra, D., & Schwing, A. G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01582 · Cited by 42
“For 2) we study Dropout [46] in the last two layers of the visual odometry model to safeguard against uncertainty within the egomotion prediction, following [25].”
MethodInfluentialIntegrity of Visual Navigation—Developments, Challenges, and Prospects
Zhu, C., Meurer, M., & Günther, C. G. (2022) · NAVIGATION Journal of the Institute of Navigation · doi:10.33012/navi.518 · Cited by 43
“In addition, the uncertainty modeling method proposed in the work by Kendall and Cipolla ( 2016 ) is also an inspiring approach to potentially provide a data-driven confidence in the output of a deep neural network, though its theoretical foundation is based on an assumed model in which the weights…”
BackgroundMethodSafe and Robust Mobile Robot Navigation in Uneven Indoor Environments
Wang, C., Wang, J., Li, C., Ho, D., Cheng, J., Yan, T., Meng, L., & Meng, M. Q. ‐. (2019) · Sensors · doi:10.3390/s19132993 · Cited by 39
“Recently, rapidly developed machine learning methods have enabled great breakthroughs in a variety of computer vision tasks and lots of efforts have been made to apply the advanced algorithms in learning-based methods to tackle the camera-based re-localization problem [21,22].”
Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering
Proenca, P. F., & Gao, Y. (2019) · arXiv · Cited by 251
“As for URSO, we collected one dataset for the dragon spacecraft and two datasets for the soyuz model with different operating ranges: soyuz_easy with [10-20] m and soyuz_hard with [10-40]”
MethodSmall Imaging Depth LIDAR and DCNN-Based Localization for Automated Guided Vehicle
Ito, S., Hiratsuka, S., Ohta, M., Matsubara, H., & Ogawa, M. (2018) · Sensors · doi:10.3390/s18010177 · Cited by 38
Drone Detection and Pose Estimation Using Relational Graph Networks
Jin, R., Jiang, J., Qi, Y., Lin, D., & Song, T. (2019) · Sensors · doi:10.3390/s19061479 · Cited by 38
Leitherer, A., Yeo, B. C., Liebscher, C. H., & Ghiringhelli, L. M. (2023) · npj Computational Materials · doi:10.1038/s41524-023-01133-1 · Cited by 39
Deep Encoder-Decoder Networks for Mapping Raw Images to Dynamic Movement Primitives
Pahič, R., Gams, A., Ude, A., & Morimoto, J. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460954 · Cited by 37
“Deep Bayesian convolutional neural networks have been applied to estimate the model’s relocalization uncertainty and improve state of the art localization accuracy [4].”
MethodLi, M., Chen, R., Liao, X., Guo, B., Zhang, W., & Guo, G. (2020) · Remote Sensing · doi:10.3390/rs12050869 · Cited by 37
Reconstruct Locally, Localize Globally: A Model Free Method for Object Pose Estimation
Cai, M., & Reid, I. D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00322 · Cited by 35
“This error is used for back-propagation to train the network, and successfully avoids the imbalanced weighting between translation and rotation when a model builds the losses using distances in the translational and rotational spaces separately (such as [24] and [23]).”
Acharya, D., Roy, S. S., Khoshelham, K., & Winter, S. (2020) · Sensors · doi:10.3390/s20195492 · Cited by 36
“Uncertainty has been modelled for CNN pose regression with Bayesian PoseNet [2] where dropout is used to draw Monte Carlo samples.”
MethodJardani, A., Vu, T. M., & Fischer, P. (2021) · Journal of Hydrology · doi:10.1016/j.jhydrol.2021.127233 · Cited by 36
Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization
Meng, L., Tung, F., Little, J. J., Valentin, J., & de Silva, C. W. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593505 · Cited by 35
“We also provide results in terms of median translation error and rotation error against Bayesian PoseNet [11], PoseNet+Geometric [12], CNN+LSTM [13], Active Search [17], SCRF [2], BTBRF [8].”
BackgroundMethodInfluentialExtending Absolute Pose Regression to Multiple Scenes
Blanton, H., Greenwell, C., Workman, S., & Jacobs, N. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00027 · Cited by 35
“A large body of work has explored methods for directly estimating camera pose from an image using convolutional neural networks [6,11,13].”
BackgroundCrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data
Qi, Y., Zheng, J., Reding, S., Li, S., & Doytchinov, I. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01684 · Cited by 34
“Absolute pose regression [29, 27, 18, 56] approaches adopt a neural network to learn implicit map representations and output the camera pose directly.”
BackgroundDong, S., Wang, S., Liu, S., Cai, L., Fan, Q., Kannala, J., & Yang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01560 · Cited by 69
“End-to-end pose regression [1, 4, 7, 18, 20–23, 29, 35, 42–44, 49, 58, 66, 69, 70, 78, 87, 89, 90, 101, 103, 104, 116, 117, 124, 129] has gained popularity due to its real-time inference capabilities.”
Network Uncertainty Informed Semantic Feature Selection for Visual SLAM
Ganti, P., & Waslander, S. L. (2019) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv.2019.00024 · Cited by 36
“Gal [26]–[29] and Kendall [30]– [32] have investigated the use of dropout [33] at test time to better estimate neural network uncertainty for both classification and regression tasks.”
BackgroundMC-CIM: Compute-in-Memory With Monte-Carlo Dropouts for Bayesian Edge Intelligence
Shukla, P., Nasrin, S., Darabi, N., Gomes, W., & Trivedi, A. R. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3224703 · Cited by 36
“the sample-ordering method, [6] shows that sufficient output statistics can be extracted from few (∼10-30) dropout iterations making pre-computation of samples and their ordering an attractive alternative for many applications.”
BackgroundMethodInfluentialScene Coordinate Regression with Angle-Based Reprojection Loss for Camera Relocalization
Li, X., Ylioinas, J., Verbeek, J. J., & Kannala, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11015-4_19 · Cited by 31
“onstrates the feasibility of directly regressing the 6 DoF camera pose from a query RGB image via a deep CNN. Later works have been proposed to extend this method for better accuracy. For example, in [17], the authors explore Bayesian Neural Networks to produce relocalization uncertainty of the predicted pose. LSTM-Po…”
MethodReliable Monte Carlo localization for mobile robots
Akai, N. (2022) · J. Field Robotics · doi:10.1002/rob.22149 · Cited by 39
“Kendall and Cipolla [Kendall and Cipolla, 2016] used this idea and showed that”
BackgroundMethodWang, S., Laskar, Z., Melekhov, I., Li, X., Zhao, Y., Tolias, G., & Kannala, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-023-01982-9 · Cited by 33
“…limitations of structure-based methods by using a neural network that directly regresses the camera pose of a query image (Kendall et al., 2015; Brahmbhatt et al., 2018; Kendall&Cipolla,2016, 2017;Melekhovetal.,2017;Walch et al., 2017; Chen et al., 2021, 2022) that is given as input to the network.”
Synchronizing Probability Measures on Rotations via Optimal Transport
Birdal, T., Arbel, M., Şimşekli, U., & Guibas, L. J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00164 · Cited by 31
“Such phenomena are ubiquitous in real scenarios and have recently lead to a paradigm shift in computer vision towards reporting the estimates in terms of either empirical or continuous multimodal probability distributions on the parameter spaces [41, 54, 14, 13], rather than single entities as done in the conventional…”
BackgroundAdaptive Leader-Follower Formation Control and Obstacle Avoidance via Deep Reinforcement Learning
Zhou, Y., Lü, F., Pu, G., Ma, X., Sun, R., Chen, H., & Li, X. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967561 · Cited by 31
“This method was later extended by introducing uncertainty modeling for factors such as noisy environments, motion blur, and silhouette lighting [6].”
RadarLoc: Learning to Relocalize in FMCW Radar
Wang, W., de Gusmão, P. P. B., Yang, B., Markham, A., & Trigoni, N. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560858 · Cited by 31
“Following variations [12], [13] improve the performance of PoseNet by introducing a geometric loss and modelling the uncertainty of poses with Bayesian Neural Network.”
MethodA 6-DOFs event-based camera relocalization system by CNN-LSTM and image denoising
Jin, Y., Yu, L., Li, G., & Fei, S. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114535 · Cited by 30
Aggressive Deep Driving: Model Predictive Control with a CNN Cost ModelPreprint
Drews, P., Williams, G., Goldfain, B., Theodorou, E. A., & Rehg, J. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05303 · Cited by 30
“An alternative method to providing absolute position uses deep neural networks to directly regress a position estimate in an area previously visited [13].”
MethodNotes on the Behavior of MC DropoutPreprint
Verdoja, F., & Kyrki, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02627 · Cited by 30
“This technique, called Monte-Carlo dropout (MCD), has proved effective to, e.g., increase visual relocalization accuracy (Kendall & Cipolla, 2016) and semantic segmentation performance on images (Kendall et al., 2017).”
Coarse-to-Fine Multi-Scene Pose Regression With Transformers
Shavit, Y., Ferens, R., & Keller, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3310929 · Cited by 29
“Various absolute pose regression methods were suggested to address these issues, proposing to modify the backbone and MLP architecture [8], [9], [10], [11], [12], as well as different loss formulations and optimization strategies [13], [14], [15].”
MethodPeretroukhin, V., Clement, L., & Kelly, J. W. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989235 · Cited by 29
Continual Learning for Image-Based Camera Localization
Wang, S., Laskar, Z., Melekhov, I., Li, X., & Kannala, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00324 · Cited by 29
Acharya, D., Tennakoon, R., Muthu, S., Khoshelham, K., Hoseinnezhad, R., & Bab‐Hadiashar, A. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104152 · Cited by 29
Relative geometry-aware siamese neural network for 6DOF camera relocalization
Li, Q., Zhu, J., Cao, R., Sun, K., Garibaldi, J. M., Li, Q., Liu, B., & Qiu, G. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.09.071 · Cited by 29
Leveraging Equivariant Features for Absolute Pose Regression
Musallam, M. A., Gaudilliere, V., Del Castillo, M. O., Al Ismaeil, K., & Aouada, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00675 · Cited by 27
“Most of the subsequent improvements lie in changes in the feature extraction architecture [3, 35, 62], modified objective functions [32, 43, 57], and additional intermediate representations [25, 27].”
BackgroundLearning Depth-Aware Deep Representations for Robotic Perception
Porzi, L., Bulò, S. R., Peñate-Sánchez, A., Ricci, E., & Moreno-Noguer, F. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2016.2637444 · Cited by 28
Active Camera Relocalization from a Single Reference Image without Hand-Eye Calibration
Tian, F.-P., Feng, W., Zhang, Q., Wang, X., Sun, J., Loia, V., & Liu, Z. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2870646 · Cited by 28
“State-of-theart SCR methods have at least cm-level translational error and degree-level angular error [21], which are too coarse to meet the above high accuracy requirement.”
Neural Refinement for Absolute Pose Regression with Feature Synthesis
Chen, S., Bhalgat, Y. S., Li, X., Bian, J.-W., Li, K., Wang, Z., & Prisacariu, V. A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01983 · Cited by 30
“2, we qualitatively compare the refinement accuracy of different APR methods - namely PoseNet[9–11], MS-Transformer[27], DFNet [7] - with our method, i.e. DFNet+NeFeS 50 .”
Calibrating Uncertainty Models for Steering Angle Estimation
Hubschneider, C., Hutmacher, R., & Zöllner, J. M. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917207 · Cited by 27
“The methods described have been shown to be applicable in camera localization [15] and semantic segmentation [16], even being able to improve model performance on a variety of tasks.”
MethodShukla, P., Muralidhar, A., Iliev, N., Tulabandhula, T., Fuller, S. B., & Trivedi, A. R. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2021.3100252 · Cited by 27
A Survey on Monocular Re-Localization: From the Perspective of Scene Map Representation
Miao, J., Jiang, K., Wen, T., Wang, Y., Jia, P., Wijaya, B., Zhao, X., Qian, C., Xiao, Z., Huang, J. H., et al. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3378716 · Cited by 27
“…(SS-APR): In order to improve the localization accuracy, contemporary APR methods achieve further improvements by developing different CNN backbones [210], [211] or MLP heads [210], modeling localization uncertainty [212], or introducing Long-Short-Term-Memory (LSTM) layers [213] in the networks.”
A review of Visual-Based Localization
Xin, X., Jiang, J., & Zou, Y. (2019) · International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI) · doi:10.1145/3366194.3366211 · Cited by 24
“However, due to the lack of sufficient training data, the pose estimated by PoseNet is less accurate than other structure-based methods.”
MethodInfluentialDo We Really Need Scene-specific Pose Encoders?
Shavit, Y., & Ferens, R. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412225 · Cited by 25
“Different variations to the encoder and MLP architectures were proposed in order to reduce overfitting [7]–[9], [11], [12].”
VNLSTM-PoseNet: A novel deep ConvNet for real-time 6-DOF camera relocalization in urban streets
Li, M., Qin, J., Li, D., Chen, R., Liao, X., & Guo, B. (2021) · Geo-spatial Information Science · doi:10.1080/10095020.2021.1960779 · Cited by 28
“Kendall and Cipolla (2016) used Bayesian CNN to estimate the uncertainty of positioning, thus improving the positioning accuracy of the system.”
MethodAkai, N., Hirayama, T., & Murase, H. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196568 · Cited by 25
“[10] also asserted that the uncertainty in pose estimation using a CNN can be evaluated if the CNN has dropout layers.”
BackgroundDiffLoc: Diffusion Model for Outdoor LiDAR Localization
Li, W., Yang, Y., Yu, S., Hu, G., Wen, C., Cheng, M., & Zang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01425 · Cited by 28
“Previous efforts [11, 12] attempt to deactivate certain neurons during inference randomly, i.e. , Monte Carlo dropout [9], and then calculate the mean and variance through multiple inferences to estimate uncertainty.”
BackgroundDeep Unsupervised Learning for Simultaneous Visual Odometry and Depth Estimation
Lu, Y., & Lu, G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803247 · Cited by 24
“For camera pose estimation, PoseNet [8] utilized deep convolutional neural networks to address 6-DoF camera pose estimation as a regression problem, which was further extended to estimate the uncertainty of predicted poses to help improve their accuracy by employing a Bayesian CNN in [9].”
MethodFeatLoc: Absolute pose regressor for indoor 2D sparse features with simplistic view synthesizing
Bui, B.-T., Dinh, T. T., & Lee, J. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.04.021 · Cited by 24
Li, Q., Cao, R., Zhu, J., Fu, H., Zhou, B., Xu, F., Jia, S., Zhang, S., Liu, K., & Li, Q. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.10.015 · Cited by 24
TransBoNet: Learning camera localization with Transformer Bottleneck and Attention
Song, X., Li, H., Li, L., Shi, W., Xie, G., Lu, X., & Hei, X. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109975 · Cited by 25
Neumann, L., & Vedaldi, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_35 · Cited by 23
“Kendall and Cipolla [15] estimate uncertainty in 6-DOF camera pose estimation by Monte Carlo sampling.”
Method6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference
Bui, M., Birdal, T., Deng, H., Albarqouni, S., Guibas, L. J., Ilić, S., & Navab, N. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_9 · Cited by 23
“This has led to a paradigm shift towards estimating a range of solutions, in the form of full probability distributions [1,7,8] or at least solutions that estimate the uncertainty in orientation estimates [43, 54].”
BackgroundMethodInfluentialCamera Pose Auto-encoders for Improving Pose Regression
Shavit, Y., & Keller, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20080-9_9 · Cited by 26
ViPR: Visual-Odometry-aided Pose Regression for 6DoF Camera Localization
Ott, F., Feigl, T., Löffler, C., & Mutschler, C. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00029 · Cited by 23
“The pioneer PoseNet [33] has been extended by Bayesian approaches [31], long short-term memories (LSTMs) [77] and others [50, 26, 36, 11].”
MethodAdversarial Networks for Camera Pose Regression and Refinement
Bui, M., Baur, C., Navab, N., Ilić, S., & Albarqouni, S. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00470 · Cited by 23
“Thus, several extensions and modifications of this method have been proposed, including uncertainty estimation [20, 21, 10], LSTM units [42], frame-to-frame information [2, 24, 13, 7] and previous pose fusion [40, 30].”
BackgroundMethodTB-Places: A Data Set for Visual Place Recognition in Garden Environments
Leyva-Vallina, M., Strisciuglio, N., López-Antequera, M., Tyleček, R., Blaich, M., & Petkov, N. (2019) · IEEE Access · doi:10.1109/access.2019.2910150 · Cited by 22
“Further modifications of PoseNet were proposed, such as a Bayesian CNN for modeling uncertainty (Kendall and Cipolla (2016)), or the addition of LSTMS for capturing feature correlation (Walch et al. (2017)).”
MethodEsfahani, M. A., Wu, K., Yuan, S., & Wang, H. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.06.014 · Cited by 21
Jo, H., & Kim, E. (2020) · IEEE Access · doi:10.1109/access.2020.2988464 · Cited by 22
“More recently, deep learning has been used for estimating absolute 6-DOF poses in an end-to-end manner [25], [29]–[32].”
MethodSynthetic-real image domain adaptation for indoor camera pose regression using a 3D model
Acharya, D., Tatli, C. J., & Khoshelham, K. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.06.013 · Cited by 22
Ren, H., Zhou, Y., Zhu, J., Lin, X., Fu, H., Huang, Y., Fang, Y., Ma, F., Yu, H., & Cheng, B. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3556561 · Cited by 41
VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization
Zhou, K., Chen, C., Wang, B., Saputra, M. R. U., Trigoni, N., & Markham, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16767 · Cited by 22
“PoseNet (Kendall, Grimes, and Cipolla 2015b), Bayesian PoseNet (Kendall and Cipolla 2016), Hourglass Network (Melekhov et al.”
BackgroundMethodTowards High-Speed Localisation for Autonomous Drone Racing
Cocoma-Ortega, J. A., & Martínez-Carranza, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33749-0_59 · Cited by 20
Lim, H., Park, C., & Myung, H. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968551 · Cited by 20
“In view of the rapid development of deep learning approaches [17], various types of deep neural architectures have been proposed for localization tasks [18]–[20].”
BackgroundEfficient Scene Compression for Visual-based Localization
Trujillo, M. M., Smith, B. C., & Fragoso, V. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00111 · Cited by 21
“The baselines for this experiment include state-of-the-art-feature-based localization methods [37, 42, 47, 50], deep-learning-based ones [22, 23, 24, 49], and state-of-the-art scene-compression algorithms [5, 8, 29].”
MethodFusing Convolutional Neural Network and Geometric Constraint for Image-Based Indoor Localization
Song, J., Patel, M., & Ghaffari, M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3140832 · Cited by 20
“The Bayesian network [10] estimates the uncertainty with its dropout approximated framework and passively “provides a measure of similarity between a testing image and the data set’s training image”.”
BackgroundMethodInfluentialAnalyzing Uncertainties in Speech Recognition Using Dropout
Vyas, A., Dighe, P., Tong, S., & Bourlard, H. A. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683086 · Cited by 20
“Computing the prediction uncertainty of a DNN model using Monte Carlo sampling with dropout has been successfully used not only to characterize model errors but also to improve the system performance in various applications [2, 3, 4, 5].”
MethodScalable Place Recognition Under Appearance Change for Autonomous DrivingPreprint
Doan, A.-D., Latif, Y., Chin, T.-J., Liu, Y., Do, T.-T., & Reid, I. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00178 · Cited by 20
“Recently, PoseNet [21] used a Convolution Neural Network (CNN) to learn this mapping, with further improvements using LSTMs to address overfitting [41], uncertainty prediction [19] and inclusion of geometric constraints [20].”
Reliability-Aware Prediction via Uncertainty Learning for Person Image Retrieval
Dou, Z., Wang, Z., Chen, W., Li, Y., & Wang, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_34 · Cited by 19
“There are mainly two types of uncertainty: data uncertainty and model uncertainty [7, 23, 40, 24].”
BackgroundImPosing: Implicit Pose Encoding for Efficient Visual Localization
Moreau, A., Gilles, T., Piasco, N., Tsishkou, D., Stanciulescu, B., & Fortelle, A. D. L. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00291 · Cited by 20
“The main advantages of this class of methods are the compatibility with real-time deployment thanks to fast inference, low memory requirements and uncertainty estimation [17, 26] which enables to filter out failure cases.”
BackgroundA Bayesian Deep Neural Network for Safe Visual Servoing in Human–Robot Interaction
Shi, L., Copot, C., & Vanlanduit, S. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.687031 · Cited by 19
“Applications include camera pose estimation (Kendall and Cipolla, 2016), depth estimation (Poggi et al., 2020), pedestrian localization (Bertoni et al., 2019), semantic segmentation (Mukhoti and Gal, 2018), and electrocardiogram signal detection (Elola et al., 2019).”
Image‐Based Indoor Localization Using Smartphone Camera
Li, S., Yu, B., Jin, Y., Huang, L. H., Zhang, H., & Liang, X. (2021) · Wireless Communications and Mobile Computing · doi:10.1155/2021/3279059 · Cited by 19
Speech Recognition: Keyword Spotting Through Image RecognitionPreprint
Gouda, S. K., Kanetkar, S., Harrison, D., & Warmuth, M. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03759 · Cited by 19
“There is some research[3, 7, 8] that uses dropout in CNNs and advices against using it.”
BackgroundOFVL-MS: Once for Visual Localization across Multiple Indoor Scenes
Xie, T., Dai, K., Lu, S., Wang, E. K., Jiang, Z., Gao, J., Liu, D., Xu, J., Zhao, L., & Li, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00508 · Cited by 18
“Current learning-based visual localization approaches can be classified into absolute pose regression (APR) [24, 55, 22], relative pose regression (RPR) [1, 11, 44], and scene coordinate regression (SCoRe) [26, 13, 61, 53, 16].”
BackgroundNigam, A., Peñate-Sánchez, A., & Agapito, L. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2858446 · Cited by 18
“Our work as another clear example of the profound impact of deep learning on robotics applications in multiple areas like recognizing change detection in SLAM [5], performing part affordance detection [4], [6] or camera re-localization [7], [8].”
BackgroundContextualNet: Exploiting Contextual Information Using LSTMs to Improve Image-Based Localization
Patel, M., Emery, B., & Chen, Y. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461124 · Cited by 18
“Abstract— Convolutional Neural Networks (CNN) have successfully been utilized for localization using a single monocular image [1].”
MethodVidLoc: 6-DoF Video-Clip Relocalization.Preprint
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · arXiv (Cornell University) · Cited by 18
Robust Six Degrees of Freedom Estimation for IIoT Based on Multibranch Network
Yang, J., Xi, M., Jiang, B., & Song, H. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.2982703 · Cited by 17
CT-Loc: Cross-domain visual localization with a channel-wise transformer
Kim, D., & Kim, J. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.014 · Cited by 18
NIDALoc: Neurobiologically Inspired Deep LiDAR Localization
Yu, S., Sun, X., Li, W., Wen, C., Yang, Y., Si, B., Hu, G., & Wang, C. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3324700 · Cited by 18
“The early variations of PoseNet propose using Bayesian CNNs to leverage the localization uncertainty [36], employing LSTM to reduce the feature dimensionality to avoid overfitting [37], or introducing learnable weight parameters to balance different losses [38].”
MethodA Simple and Effective Point-Based Network for Event Camera 6-DOFs Pose Relocalization
Ren, H., Zhu, J., Zhou, Y., Fu, H., Huang, Y., & Cheng, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01715 · Cited by 19
“Moreover, we present results derived from other well-established computer vision methods, including PoseNet[14], Bayesian PoseNet [13], Pairwise-CNN [15], LSTM-Pose [39], and SP-LSTM[26].”
MethodGeoPose3K: Mountain landscape dataset for camera pose estimation in outdoor environments
Brejcha, J., & Čadík, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.05.009 · Cited by 17
“The topic gained a lot of interest in the context of urban environments, where imagebased methods proved to be a successful solution [21, 22, 23, 24, 25, 26, 27, 28, 29, 30].”
BackgroundAutomatic Registration of Images to Untextured Geometry Using Average Shading Gradients
Plötz, T., & Roth, S. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1022-x · Cited by 17
“As shown by Kendall et al. (2015), Kendall and Cipolla (2016) it is also possible to directly regress the 6D camera pose from a single image using a CNN.”
A Systematic Approach to Analyzing Perception Architectures in Autonomous Vehicles
Kurzidem, I., Saad, A., & Schleiß, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58920-2_10 · Cited by 17
Learning to Localize in New Environments from Synthetic Training Data
Winkelbauer, D., Denninger, M., & Triebel, R. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560872 · Cited by 17
“Bayesian PoseNet [25] is trained via monte carlo dropout which we also use in our network to improve the triangulation step.”
MethodMüller, M. S., Urban, S., & Jutzi, B. (2017) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-annals-iv-2-w3-49-2017 · Cited by 17
“An enhancement is the Bayesian PoseNet (Kendall and Cipolla, 2016) which provides re-localization uncertainty by adding dropout layers after each convolutional layer and improves the PoseNet accuracy by averaging over multiple forward passes.”
MethodBAR: Bayesian Activity Recognition using variational inferencePreprint
Krishnan, R., Subedar, M. M., & Tickoo, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03305 · Cited by 20
“Bayesian deep learning framework with Monte Carlo (MC) dropout technique has been used in visual scene understanding applications including camera relocalization [16], semantic segmentation [17], and depth regression [18].”
MethodBenchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection TasksPreprint
Band, N., Rudner, T. G. J., Feng, Q., Filos, A., Nado, Z., Dusenberry, M. W., Jerfel, G., Tran, D., & Gal, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.12717 · Cited by 36
Gundersen, K., Alendal, G., Oleynik, A., & Blaser, N. (2020) · Algorithms · doi:10.3390/a13060145 · Cited by 15
An Accurate, Robust Visual Odometry and Detail-Preserving Reconstruction System
Gong, X., Liu, Y., Wu, Q., Huang, J., Zong, H., & Wang, J. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3017886 · Cited by 16
“instance, PoseNet [27] and [28] uses an end-to-end manner to achieve 6D relocalization.”
STUN: Self-Teaching Uncertainty Estimation for Place Recognition
Cai, K., Lu, C. X., & Huang, X. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981546 · Cited by 16
“During the inference phase, forward propagation is performed 40 times as suggested by [21].”
BackgroundMethodWang, J., Yu, H., Lin, X., Li, Z., Sun, W. J., & Akhtar, N. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122755 · Cited by 16
ENG: End-to-End Neural Geometry for Robust Depth and Pose Estimation Using CNNs
Dharmasiri, T., Spek, A., & Drummond, T. W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20887-5_39 · Cited by 16
“ghlighted the increased difficulty of predicting 3D quantities over 2D quantities. Kendall and Cipolla demonstrated that camera pose prediction from a single image catered for relocalization scenarios [16]. However, each of the above works lack a representation of structure as they do not explicitly predict depths. Our…”
Result
Metadata
- Authors
- Kendall, A., & Cipolla, R.
- Affiliations
- University of Cambridge (GB)
- ORCID
- Cipolla, Roberto
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 4762-4769
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2279895976
- Cited by
- 548
- References
- 30
- Citations by year
- 2026: 20 · 2025: 32 · 2024: 44 · 2023: 57 · 2022: 52 · 2021: 68 · 2020: 90 · 2019: 82 · 2018: 57 · 2017: 39 · 2016: 6
- FWCI
- 107.26
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- camera relocalization, Bayesian convolutional neural network, uncertainty estimation, 6D pose estimation, real-time localization, out-of-distribution detection, indoor and outdoor scenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
