Modelling uncertainty in deep learning for camera relocalization
Open accessAbstract
We present a robust and real-time monocular six degree of freedom visual relocalization system. We use a Bayesian convolutional neural network to regress the 6-DOF camera pose from a single RGB image. It is trained in an end-to-end manner with no need of additional engineering or graph optimisation. The algorithm can operate indoors and outdoors in real time, taking under 6ms to compute. It obtains approximately 2m and 6° accuracy for very large scale outdoor scenes and 0.5m and 10° accuracy indoors. Using a Bayesian convolutional neural network implementation we obtain an estimate of the model's relocalization uncertainty and improve state of the art localization accuracy on a large scale outdoor dataset. We leverage the uncertainty measure to estimate metric relocalization error and to detect the presence or absence of the scene in the input image. We show that the model's uncertainty is caused by images being dissimilar to the training dataset in either pose or appearance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ResNet architectures with pre-trained weights have low number of pooling layers and good accuracy-complexity trade-off.
“We adopted the ResNet architectures with pre-trained weights as the network backbone, due to its low number of pooling layers, and good accuracy-complexity trade-off [35].”
From Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering (2020) · cites this paper as [35]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1× · see citing works
“Convolutional neural networks have recently been shown to be extremely powerful for generating image representations for classification [12], [13].”
BackgroundDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 3× · see citing works
“This has been achieved using SIFT [2] features with FAB-MAP [6] or using convolutional neural network features [7].”
BackgroundMethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 4× · see citing works
“Additionally, in green we show the output from the first auxiliary pose regressor from the GoogLeNet architecture (see Figure 3 of [13]).”
BackgroundMethodInfluential(2014) · Cited by 33,419 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1× · see citing works
“These higher level contours and shape cues [14] have been shown to be increasingly invariant to object instantiation parameters.”
BackgroundCaffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1× · see citing works
“Using standard libraries such as [25] we can obtain an efficient implementation of a Bernoulli Bayesian convolutional neural network.”
MethodORB: An efficient alternative to SIFT or SURF
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126544 · Cited by 11,037 · cited 2× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundMethodThrun, S. (2002) · CACM · doi:10.1145/504729.504754 · Cited by 7,941 · cited 1× · see citing works
“nsider localization uncertainty [1], [5], [11], previous probabilistic algorithms have been proposed. Bayesian approaches include extended Kalman filters and particle filter approaches such as FastSLAM [18]. However these approaches estimate uncertainty from sensor noise models, not the uncertainty of the model to repre…”
MethodORB-SLAM: A Versatile and Accurate Monocular SLAM System
Mur-Artal, R., Montiel, J. M. M., & Tardos, J. D. (2015) · IEEE Transactions on Robotics · doi:10.1109/tro.2015.2463671 · Cited by 6,778 · cited 1× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundDropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · cited 1× · see citing works
“Recent work [23] has extended dropout to approximate Bayesian inference over the network’s weights.”
BackgroundHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1× · see citing works
“[28] argues that transferring weights can cause performance to drop in two situations.”
BackgroundCaffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,749 · cited 1× · see citing works
“Using standard libraries such as [25] we can obtain an efficient implementation of a Bernoulli Bayesian convolutional neural network.”
MethodParallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · cited 8× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundInfluentialLSD-SLAM: Large-scale direct monocular SLAM
Engel, J., Schöps, T., & Cremers, D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10605-2_54 · Cited by 3,851 · cited 3× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundDeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.214 · Cited by 3,288 · cited 1× · see citing works
“However their use for regression has been predominantly limited to object detection [13] and human pose regression [15].”
BackgroundA practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · cited 1× · see citing works
“Neural networks which consider uncertainty are known as Bayesian neural networks [19], [20].”
Background(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534 · see citing works
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489 · see citing works
Self-citationDTAM: Dense tracking and mapping in real-time
Newcombe, R. A., Lovegrove, S. J., & Davison, A. J. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126513 · Cited by 2,078 · cited 2× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundVariational inference for Dirichlet process mixtures
Jordan, M. I., & Jordan, M. I. (2006) · Bayesian Analysis · doi:10.1214/06-ba104 · Cited by 1,645 · cited 1× · see citing works
“However we found that all of these methods produced poorer localization uncertainty than fitting a single unimodal Gaussian to the data.”
MethodFAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance
Cummins, M., & Newman, P. (2008) · The International Journal of Robotics Research · doi:10.1177/0278364908090961 · Cited by 1,472 · cited 1× · see citing works
“This has been achieved using SIFT [2] features with FAB-MAP [6] or using convolutional neural network features [7].”
MethodPractical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079 · see citing works
Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., & Fitzgibbon, A. W. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.377 · Cited by 952 · cited 7× · see citing works
“A learning approach to relocalization has also been proposed previously using random forests to regress Scene Coordinate labels [16].”
MethodInfluentialBayesian Convolutional Neural Networks with Bernoulli Approximate Variational InferencePreprint
Gal, Y., & Ghahramani, Z. (2015) · arXiv.org · doi:10.48550/arxiv.1506.02158 · Cited by 839 · cited 7× · see citing works
“The results in Table 6 show that using Monte Carlo dropout [24] results in a considerable improvement in localization accuracy over PoseNet [8].”
BackgroundMethodResultInfluentialPlace Recognition with ConvNet Landmarks: Viewpoint-Robust, Condition-Robust, Training-Free
Sünderhauf, N., Rowlands, S., Jacobson, A., Dayoub, F., Pepperell, E., Upcroft, B., & Milford, M. J. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.022 · Cited by 374 · see citing works
Worldwide Pose Estimation Using 3D Point Clouds
Li, Y., Snavely, N., Huttenlocher, D. P., & Fua, P. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_2 · Cited by 369 · cited 3× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundTransforming Neural-Net Output Levels to Probability Distributions
Denker, J. S., & LeCun, Y. (1990) · Cited by 223 · see citing works
Exploiting uncertainty in regression forests for accurate camera relocalization
Valentin, J., Niebner, M., Shotton, J., Fitzgibbon, A., Izadi, S., & Torr, P. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299069 · Cited by 156 · cited 1× · see citing works
“It was extended with a probabilistic approach in [17].”
Coarse-to-fine vision-based localization by indexing scale-Invariant features
Wang, J., Zha, H., & Cipolla, R. (2006) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859085 · Cited by 136 · cited 1× · see citing works
“Visual SLAM [1], [4], [5], [9], and relocalization research [10], [11], has largely focused on registering viewpoints using point-based landmark features, such as SIFT [2] or ORB [3].”
BackgroundSelf-citationConvolutional networks for real-time 6-DOF camera relocalization.Preprint
Kendall, A., Grimes, M. K., & Cipolla, R. (2015) · arXiv (Cornell University) · Cited by 50 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
“uncertainty quantity technique [397], [43], [249], [420], [394], [413], [10], [411], [418], applicable on various deep learning architectures.”
BackgroundInfluentialPoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489 · see citing works
Self-citationKendall, A., Badrinarayanan, V., & Cipolla, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.57 · Cited by 784 · see citing works
“This technique has seen success in modelling uncertainty for camera relocalisation [19].”
MethodSelf-citationGeometric Loss Functions for Camera Pose Regression with Deep Learning
Kendall, A., & Cipolla, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.694 · Cited by 803 · see citing works
“Later work has extended PoseNet to use RGB-D input [25], learn relative ego-motion [31], improve the context of features [54], localise over video sequences [8] and interpret relocalisation uncertainty with Bayesian Neural Networks [19].”
BackgroundMethodInfluentialSelf-citationDeMoN: Depth and Motion Network for Learning Monocular Stereo
Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.596 · Cited by 777 · see citing works
“[21] trained a ConvNet for camera relocalization — predicting the location of the camera within a known scene from a single image.”
BackgroundLeveraging uncertainty information from deep neural networks for disease detection
Leibig, C., Allken, V., Ayhan, M. S., Berens, P., & Wahl, S. (2017) · Scientific Reports · doi:10.1038/s41598-017-17876-z · Cited by 510 · see citing works
Geometry-Aware Learning of Maps for Camera Localization
Brahmbhatt, S., Gu, J., Kim, K., Hays, J. H., & Kautz, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00277 · Cited by 421 · see citing works
“Our work is built upon prior works in DNN-based pose estimation methods [11, 30, 31, 32, 40, 55], which regress 6-DoF camera pose from an input RGB image with a DNN.”
BackgroundMethodInfluentialUnderstanding the Limitations of CNN-Based Absolute Camera Pose Regression
Sattler, T., Zhou, Q., Pollefeys, M., & Leal-Taixe, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00342 · Cited by 418 · see citing works
“[12, 28] also reason about the uncertainty of the estimated poses.”
BackgroundImage-Based Localization Using LSTMs for Structured Feature Correlation
Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.75 · Cited by 521 · see citing works
“Scene Area or Volume Active Search (w/o) [43] Active Search (w/) [43] PoseNet [23] Bayesian PoseNet [22] Proposed + Improvement(pos,ori)”
BackgroundMethodResultInfluentialA Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving
Feng, D., Harakeh, A., Waslander, S. L., & Dietmayer, K. (2020) · IEEE transactions on intelligent transportation systems (Print) · doi:10.1109/tits.2021.3096854 · Cited by 328 · see citing works
“…in different components of autonomous driving such as semantic segmentation [36]–[38], optical flow [39], [40], depth estimation [35], [41]–[43], visual odometry [44]– [46], image annotation [47]–[50], visual tracking [51], trajectory prediction [52]–[55] and end-to-end perception [56]–[58].”
BackgroundChen, Y., Schmid, C., & Sminchisescu, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00716 · Cited by 264 · see citing works
“Several methods train networks based on ground-truth labels and have been successfully applied to many tasks, such as monocular depth prediction [9, 10, 27, 28], optical flow [8, 19, 35, 38] and camera pose estimation [22, 23, 24].”
MethodDeep Auxiliary Learning for Visual Localization and Odometry
Valada, A., Radwan, N., & Burgard, W. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462979 · Cited by 260 · see citing works
“We compare the performance of our VLocNet architecture with current state-of-the-art deep learning-based localization methods namely PoseNet [12], Bayesian PoseNet [10], LSTM-Pose [26], VidLoc [4], Hourglass-Pose [16], BranchNet [27], PoseNet2 [11], SVS-Pose [18], and NNnet [14].”
BackgroundMethodInfluentialArtificial intelligence (AI) in augmented reality (AR)-assisted manufacturing applications: a review
Sahu, C. K., Young, C., & Rai, R. (2020) · International Journal of Production Research · doi:10.1080/00207543.2020.1859636 · Cited by 271 · see citing works
“Efforts at increasing the accuracy of PoseNet included incorporating a geometric loss function (Kendall and Cipolla 2017) and a Bayesian CNN (MacKay 1992) to model the uncertainty (Kendall and Cipolla 2016).”
MethodDropout Sampling for Robust Object Detection in Open-Set Conditions
Miller, D., Nicholson, L., Dayoub, F., & Sünderhauf, N. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460700 · Cited by 257 · see citing works
“y deep neural network to become Bayesian by simply enabling the dropout layers during testing, as opposed to standard practice where dropout layers are only used during training. Recently, in [8] and [9], dropout sampling was used for uncertainty estimates on regression and image classification tasks in order to improv…”
MethodInfluentialVidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.284 · Cited by 249 · see citing works
“For this reason, we In [8], one possible means of representing uncertainty in the global pose estimation was considered.”
BackgroundConcrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning
McAllister, R., Gal, Y., Kendall, A., van der Wilk, M., Shah, A., Cipolla, R., & Weller, A. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/661 · Cited by 245 · see citing works
Self-citationA survey on Visual-Based Localization: On the benefit of heterogeneous data
Piasco, N., Désiré Sidibé, A., Demonceaux, C., & Gouet-Brunet, V. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.09.013 · Cited by 238 · see citing works
“Recent improvement have been proposed by the original authors [78] to integrate an uncertainty estimation in the regression process.”
BackgroundInfluentialEnd-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks
Wang, S., Clark, R., Wen, H.-K., & Trigoni, N. (2018) · Int. J. Robotics Res. · doi:10.1177/0278364917734298 · Cited by 224 · see citing works
“The technique has also been applied to improve camera relocalization (Kendall and Cipolla, 2016) and image segmentation (Kendall et al.”
MethodVLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry
Radwan, N., Valada, A., & Burgard, W. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2869640 · Cited by 237 · see citing works
“PoseNet [13] Bayesian PoseNet [29] SVSPose [4] VLocNet [5] VLocNet++STL (Ours)”
MethodRelocNet: Continuous Metric Learning Relocalisation Using Neural Nets
Balntas, V., Li, S., & Prisacariu, V. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_46 · Cited by 231 · see citing works
“Convolutional neural network methods, starting with PoseNet [25], regress camera poses from single RGB images.”
MethodInfluentialDeep Learning for Spacecraft Pose Estimation from Photorealistic Rendering
Proenca, P. F., & Gao, Y. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197244 · Cited by 229 · see citing works
“As for URSO, we collected one dataset for the dragon spacecraft and two datasets for the soyuz model with different operating ranges: soyuz_easy with [10-20] m and soyuz_hard with [10-40]”
Method3D Pose Regression Using Convolutional Neural Networks
Mahendran, S., Ali, H., & Vidal, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.73 · Cited by 210 · see citing works
“Quaternions have also been used to represent 3D pose for camera localization in [11, 10, 9], but these works ignore the unit-norm constraint for computational ease and use a mean-squared or reprojection loss, whereas we incorporate the constraint into the network and use a geodesic loss.”
MethodImage-Based Localization Using Hourglass Networks
Melekhov, I., Ylioinas, J., Kannala, J., & Rahtu, E. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.107 · Cited by 208 · see citing works
“In general, HourglassSum-Pose improves the accuracy of the camera position by 52.27% and orientation by 8.47% for average error with respect to PoseNet.”
BackgroundMethodInfluentialNeural Network-Based Pose Estimation for Noncooperative Spacecraft Rendezvous
Sharma, S., & D'Amico, S. (2020) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2020.2999148 · Cited by 196 · see citing works
“Unlike state-of-the-art methods such as the probabilistic PoseNet [69], the SPN method can compute this uncertainty without requiring multiple Monte-Carlo samplings of its CNN.”
MethodSatellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement
Chen, B., Cao, J., Parra, A., & Chin, T.-J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00343 · Cited by 203 · see citing works
“End-to-end learning The success of deep learning in image classification and object detection has motivated a large number of works on end-to-end learning for pose estimation [34, 7, 15, 16, 17, 23].”
BackgroundPerez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F. J., Englund, C., Tauber, M., Nikolakopoulos, G., & Flores, J. L. (2024) · ACM Computing Surveys · doi:10.1145/3626314 · Cited by 185 · see citing works
AtLoc: Attention Guided Camera Localization
Wang, B., Chen, C., Lu, C. X., Zhao, P., Trigoni, N., & Markham, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6608 · Cited by 179 · see citing works
“Previous works(Kendall and Cipolla 2017; Brahmbhatt et al. 2018) showed successful applications of the classical convolutional neural network (CNN) architectures in camera pose estimation, e.g. GoogleNet (Szegedy et al. 2015) and ResNet (He et al. 2016).”
MethodIndoor Localization Improved by Spatial Context—A Survey
Gu, F., Hu, X., Ramezani, M., Acharya, D., Khoshelham, K., Valaee, S., & Shang, J. (2019) · ACM Computing Surveys · doi:10.1145/3322241 · Cited by 170 · see citing works
“Parametric approaches train a model (a classifier or regressor) using geo-tagged images and predict the corresponding location of the query image using the trained model [68, 171].”
MethodLearning to Navigate the Energy Landscape
Valentin, J., Dai, A., Nießner, M., Kohli, P., Torr, P. H. S., Izadi, S., & Keskin, C. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.41 · Cited by 155 · see citing works
“We also demonstrate results on RGB camera relocalization, and refer the reader to [22] for a tour of the literature.”
BackgroundLearning Multi-Scene Absolute Pose Regression with Transformers
Shavit, Y., Ferens, R., & Keller, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00273 · Cited by 152 · see citing works
“Various absolute pose regression methods were suggested to address these issues, proposing modifications to the backbone and MLP architecture [20, 21, 35, 37, 29, 36, 8], as well as different loss formulations and optimization strategies [15, 16, 28].”
MethodHierarchical Scene Coordinate Classification and Regression for Visual Localization
Li, X., Wang, S., Zhao, Y., Verbeek, J. J., & Kannala, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01200 · Cited by 149 · see citing works
“PoseNet [24] and its variants [9, 22, 23, 31, 58] directly regress the camera pose from a query image.”
BackgroundCamNet: Coarse-to-Fine Retrieval for Camera Re-Localization
Ding, M., Wang, Z., Sun, J., Shi, J., & Luo, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00296 · Cited by 145 · see citing works
“Inspired by the success of deep learning models, a series of approaches [17, 41, 15, 16, 7, 38, 39] have been proposed to regress the absolute camera pose directly from RGB images.”
BackgroundMethodAlizadehsani, R., Roshanzamir, M., Hussain, S., Khosravi, A., Koohestani, A., Zangooei, M. H., Abdar, M., Beykikhoshk, A., Shoeibi, A., Zare, A., et al. (2021) · Annals of Operations Research · doi:10.1007/s10479-021-04006-2 · Cited by 153 · see citing works
“Kendall et al. (Kendall and Gal 2017) focused on epistemic uncertainty in computer vision using Bayesian deep learning models.”
InfluentialDelving deeper into convolutional neural networks for camera relocalization
Wu, J., Ma, L., & Hu, X. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989663 · Cited by 143 · see citing works
“One limitation of our approach, as well as PoseNet approaches [7], [8], is that it is only suitable for scenarios while depth information is unavailable because when depth information is available, there exist much more accurate approaches, for example, SCoRe Forests [2].”
BackgroundMethodInfluentialConcrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning
McAllister, R., Gal, Y., Kendall, A., van der Wilk, M., Shah, A., Cipolla, R., & Weller, A. (2017) · Cambridge University Engineering Department Publications Database · doi:10.17863/cam.12760 · Cited by 143 · see citing works
“For this task as well there exist computer vision techniques which use Bayesian deep learning [Kendall and Cipolla, 2016].”
MethodSelf-citationDeep regression for monocular camera-based 6-DoF global localization in outdoor environments
Naseer, T., & Burgard, W. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205957 · Cited by 141 · see citing works
“Inspired by the amazing ability of such networks to perform well under harsh visual conditions, Kendall and Cipolla [7, 8] (Bayesian PoseNet, PoseNet) have explored the area of directly regressing the camera pose from these networks.”
BackgroundDeepSLAM: A Robust Monocular SLAM System With Unsupervised Deep Learning
Li, R., Wang, S., & Gu, D. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.2982096 · Cited by 135 · see citing works
“order to estimate the uncertainty of pose estimation, Kendall and Cipolla [14] proposed Bayesian PoseNet by considering”
MethodInfluentialΛαμπρόπουλος, Γ., Keramopoulos, E., & Diamantaras, K. (2020) · Visual Informatics · doi:10.1016/j.visinf.2020.01.001 · Cited by 139 · see citing works
Dropout Inference in Bayesian Neural Networks with Alpha-divergencesPreprint
Li, Y.-Z., & Gal, Y. (2017) · International Conference on Machine Learning · doi:10.48550/arxiv.1703.02914 · Cited by 246 · see citing works
“These have been used for machine vision and medical applications (Kendall & Cipolla, 2016; Kendall et al., 2015; Angermueller & Stegle, 2015; Yang et al., 2016).”
MethodKarimi, D., Zeng, Q., Mathur, P., Avinash, A., Mahdavi, S. S., Spadinger, I., Abolmaesumi, P., & Salcudean, S. E. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.07.005 · Cited by 127 · see citing works
Efficient Uncertainty Estimation for Semantic Segmentation in Videos
Huang, P., Hsu, W., Chiu, C.-Y., Wu, T., & Sun, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_32 · Cited by 123 · see citing works
“This technique called MC dropout that has been successfully used in camera relocalisation [21] and segmentation [20].”
MethodAcharya, D., Khoshelham, K., & Winter, S. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.02.020 · Cited by 114 · see citing works
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12567 · Cited by 130 · see citing works
“…[236] Global optimization [64], [123], [237], [238] Keyframe detection [77] Loop-closure detection [239], [240], [241], [242] Uncertainty Estimation [135], [137], [243] integrates a differentiable second-order optimizer (LM al-gorithm) into a deep neural network to achieve end-to-end learning.”
BackgroundMethodEvaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detection
Miller, D., Dayoub, F., Milford, M. J., & Sünderhauf, N. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793821 · Cited by 108 · see citing works
“MC Dropout has previously been used to estimate the epistemic uncertainty for regression, image classification and segmentation [20]–[22].”
MethodEnd-to-End Learnable Geometric Vision by Backpropagating PnP Optimization
Chen, B., Parra, Á., Cao, J., Li, N., & Chin, T.-J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00812 · Cited by 107 · see citing works
“On the other extreme, end-to-end methods have been devised [23, 21, 22, 8, 32, 50, 52, 9] that bypass geometric optimization, by using fully connected layers to compute the geometric quantity (e.g., 6DOF camera pose) from a feature map derived from previous layers.”
BackgroundTo Learn or Not to Learn: Visual Localization from Essential Matrices
Zhou, Q., Sattler, T., Pollefeys, M., & Leal-Taixé, L. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196607 · Cited by 105 · see citing works
“We compare our approach to methods for absolute pose regression (APR) [9, 30–32, 77] As can be seen in Tab.”
BackgroundMethodInfluentialCMRNet: Camera to LiDAR-Map Registration
Cattaneo, D., Vaghi, M., Ballardini, A. L., Fontana, S., Sorrenti, D. G., & Burgard, W. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917470 · Cited by 99 · see citing works
“Starting from this work, additional improvements have been proposed by introducing new geometric loss functions [15], by exploiting the uncertainty estimation of Bayesian CNNs [16], by including a data augmentation scheme based on synthetic depth information [17], or using the relative pose between two observations in…”
MethodDeep Learning for Visual Localization and Mapping: A Survey
Chen, C., Wang, B., Lu, C. X., Trigoni, N., & Markham, A. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3309809 · Cited by 97 · see citing works
“…units to reduce the dimensionality [74], applying synthetic generation to augment training data [70], [73], [77], [97], replacing the backbone [75], modeling pose uncertainty [69], [78], [98], introducing geometry-aware loss function [72], and associating features via an attention mechanism [82].”
Real-Time 6DOF Pose Relocalization for Event Cameras With Stacked Spatial LSTM Networks
Nguyen, A., Do, T.-T., Caldwell, D. G., & Tsagarakis, N. G. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00207 · Cited by 92 · see citing works
DFNet: Enhance Absolute Pose Regression with Direct Feature Matching
Chen, S., Li, X., Wang, Z., & Prisacariu, V. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20080-9_1 · Cited by 91 · see citing works
Real-Time RGB-D Camera Pose Estimation in Novel Scenes Using a Relocalisation Cascade
Cavallari, T., Golodetz, S., Lord, N. A., Valentin, J., Prisacariu, V. A., Stefano, L. D., & Torr, P. H. S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2915068 · Cited by 89 · see citing works
“this to model the uncertainty in the result [39] and explore different loss functions to achieve better results [40].”
BackgroundUncertainty-Aware Audiovisual Activity Recognition Using Deep Bayesian Variational Inference
Subedar, M., Krishnan, R., Meyer, P. L., Tickoo, O., & Huang, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00640 · Cited by 85 · see citing works
“Bayesian deep learning framework with Monte Carlo (MC) dropout approximate inference [13] is used in visual scene understanding applications including camera relocalization [25], semantic segmentation [24] and depth regression [26].”
MethodKFNet: Learning Temporal Camera Relocalization Using Kalman Filtering
Zhou, L., Luo, Z., Shen, T., Zhang, J., Zhen, M., Yao, Y., Fang, T., & Quan, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00497 · Cited by 84 · see citing works
“Taking the negative logarithm of the probability density function (PDF) of θ(i), we define the loss based on the likelihood which gives rise to the maximum likelihood (ML) estimation for each pixel in the form [23]:”
BackgroundMethodInfluentialA survey on epistemic (model) uncertainty in supervised learning: Recent advances and applications
Zhou, X., Liu, H., Pourpanah, F., Zeng, T., & Wang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.119 · Cited by 81 · see citing works
“…vision, uncertainty is taken into account in variety of applications such as image classification [151, 152], segmentation [83, 153], camera relocalization [154], object detection [155, 156, 157], image/video retrieval (restoration) [158, 159], in the setting of Bayesian and ensemble learning.”
BackgroundDeep Learning Techniques for Visual SLAM: A Survey
Mokssit, S., Licea, D. B., Guermah, B., & Ghogho, M. (2023) · IEEE Access · doi:10.1109/access.2023.3249661 · Cited by 81 · see citing works
“In the visual SLAM context, [145] uses a bayesian convolutional neural network to regress camera 6-DOF poses directly from raw RGB images.”
MethodOmnidirectional CNN for Visual Place Recognition and Navigation
Wang, T.-H. J., Huang, H.-J., Lin, J.-T., Hu, C.-W., Zeng, K.-H., & Sun, M. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463173 · Cited by 79 · see citing works
“directly regress the 6-DoF camera pose from the observation to obtain absolute position [11], [22], [25], [26], [27].”
BackgroundDeep Predictive Models for Collision Risk Assessment in Autonomous Driving
Strickland, M., Fainekos, G., & Amor, H. B. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461160 · Cited by 78 · see citing works
“Others have also employed Bayesian inference techniques to determine model uncertainty and used the results to improve understanding of driving scenarios [12] and reduce SLAM relocalization errors [13].”
MethodLeveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection
Feng, D., Rosenbaum, L., Timm, F., & Dietmayer, K. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814046 · Cited by 79 · see citing works
“It can be quantified through variational inference [23] or the sampling techniques [13], [24], and has been applied to active learning [25], [26], image semantic segmentation [27], [28], camera location estimation [29] or open-dataset object detection problems [30].”
Background3D convolutional neural network for object recognition: a review
Singh, R. D., Mittal, A., & Bhatia, R. K. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6912-6 · Cited by 78 · see citing works
“Cambridge landmarks The Cambridge Landmarks dataset [57] addresses a sizeable urban localisation dataset of 6 scenes captured in Cambridge University with 6-DOF camera pose.”
BackgroundLearning Multi-View Camera Relocalization With Graph Neural Networks
Xue, F., Wu, X., Cai, S., & Wang, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01139 · Cited by 78 · see citing works
“Both image-based [19, 17, 18, 25, 5, 46, 44, 42, 16, 4]) and sequence-based algorithms (VidLoc [9], MapNet [3], and LsG [50]) are considered.”
BackgroundMethodInfluentialEvaluating Uncertainty Quantification in End-to-End Autonomous Driving ControlPreprint
Michelmore, R., Kwiatkowska, M., & Gal, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06817 · Cited by 77 · see citing works
“In 2016, Kendall and Cipolla [15] developed tools for the localisation of a car given a forward facing photo.”
BackgroundMethodRandom forests versus Neural Networks — What's best for camera localization?
Massiceti, D., Krull, A., Brachmann, E., Rother, C., & Torr, P. H. S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989598 · Cited by 73 · see citing works
“Inspired by successes in deep learning for a multitude of tasks, [17, 18] were first to propose the use of a deep CNN as an end-to-end 6D camera pose regressor using RGB and RGB-D images as input.”
Indoor Relocalization in Challenging Environments With Dual-Stream Convolutional Neural Networks
Li, R., Liu, Q., Gui, J., Gu, D., & Hu, H. (2017) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2017.2664920 · Cited by 74 · see citing works
“We also transform the dual-stream CNN to a Bayesian dual-stream CNN and compare it with the Bayesian PoseNet [25].”
BackgroundMethodInfluentialOngoing Evolution of Visual SLAM from Geometry to Deep Learning: Challenges and Opportunities
Li, R., Wang, S., & Gu, D. (2018) · Cognitive Computation · doi:10.1007/s12559-018-9591-8 · Cited by 76 · see citing works
“[64] further proposed Bayesian PoseNet by using dropout layers in the network as a means of sampling.”
Geometric Loss Functions for Camera Pose Regression with Deep LearningPreprint
Kendall, A. G., & Cipolla, R. (2017) · arXiv (Cornell University) · doi:10.17863/cam.26496 · Cited by 84 · see citing works
Self-citationScalable Place Recognition Under Appearance Change for Autonomous Driving
Doan, A.-D., Latif, Y., Chin, T.-J., Liu, Y., Do, T.-T., & Reid, I. D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00941 · Cited by 74 · see citing works
“Recently, PoseNet [21] used a Convolution Neural Network (CNN) to learn this mapping, with further improvements using LSTMs to address overfitting [41], uncertainty prediction [19] and inclusion of geometric constraints [20].”
Looking to Relations for Future Trajectory Forecast
Choi, C., & Dariush, B. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00101 · Cited by 75 · see citing works
“Dropout variational inference is a practical technique [12, 13] to approximate variational inference using dropout at training time to update model parameters and at test time to sample from the dropout distribution q(ω).”
MethodRETRACTED ARTICLE: A review on quantum computing and deep learning algorithms and their applications
Valdez, F., & Melín, P. (2022) · Soft Computing · doi:10.1007/s00500-022-07037-4 · Cited by 75 · see citing works
“We found 1160 linked papers considering Kendall and Cipolla (2016) 239 A review on quantum computing and deep learning algorithms and their applications title, abstract and keyword.”
BackgroundRetractedDirect-PoseNet: Absolute Pose Regression with Photometric Consistency
Chen, S., Wang, Z., & Prisacariu, V. A. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00125 · Cited by 72 · see citing works
“Prior works [6, 20, 21, 22, 30, 43, 58, 59] typically use a quaternion or an axis-angle representation during pose estimation, requiring balancing between rotation and translation terms.”
BackgroundMethodA Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy TasksPreprint
Filos, A., Farquhar, S., Gomez, A. N., Rudner, T. G. J., Kenton, Z., Smith, L., Alizadeh, M., de Kroon, A., & Gal, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10481 · Cited by 69 · see citing works
“BDL has already been demonstrated to play a crucial role in applications such as medical diagnostics [30, 16, 5, 49] (see Figure 1), computer vision [21, 18, 17], in the sciences [31, 34], and autonomous driving [1, 15, 21, 18, 19].”
BackgroundRelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild
Zhang, J. Y., Ramanan, D., & Tulsiani, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19821-2_34 · Cited by 70 · see citing works
Hybrid Scene Compression for Visual Localization
Camposeco, F., Cohen, A., Pollefeys, M., & Sattler, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00784 · Cited by 68 · see citing works
“To this end, we compare to state-ofthe-art feature-based methods [8, 18, 34], as well as recent deep-learning-based methods [16, 17, 20, 21], as they also reduce the memory consumption of the localization database w.”
MethodInfluentialPointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization
Wang, W., Wang, B., Zhao, P., Chen, C., Clark, R., Yang, B., Markham, A., & Trigoni, N. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3128683 · Cited by 62 · see citing works
“For example, PoseNet related works [10], [11], [18] proved the feasibility of predicting the global pose using a single RGB image by regressing the pose directly.”
BackgroundLocal Supports Global: Deep Camera Relocalization With Sequence Enhancement
Xue, F., Wang, X., Yan, Z., Wang, Q., Wang, J., & Zha, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00293 · Cited by 62 · see citing works
“Baseline Methods When evaluating on the 7-Scenes benchmark, the baseline algorithms include PoseNet15 [16], PoseNet16 [14], PoseNet17 [15], Hourglass [20], LSTM-PoseNet [34], VidLoc [7], and MapNet [4].”
BackgroundMethodIndoor Visual Positioning Aided by CNN-Based Image Retrieval: Training-Free, 3D Modeling-Free
Chen, Y., Chen, R., Liu, M., Xiao, A., Wu, D., & Zhao, S. (2018) · Sensors · doi:10.3390/s18082692 · Cited by 62 · see citing works
“PoseNet was the first approach to use DCNNs to solve the metric localization problem, and then Bayesian CNN implementation was utilized to address the pose uncertainty [39].”
MethodRPNet: An End-to-End Network for Relative Camera Pose Estimation
En, S., Lechervy, A., & Jurie, F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11009-3_46 · Cited by 59 · see citing works
“k for camera pose estimation from single RGB images is PoseNet [9]. It is based on GoogLeNet with two output branches to regress translations and rotations. PoseNet followup includes: Baysian PoseNet [7], Posenet-LSTM [20] where LSTM is used to model the context of the images, Geometric-PoseNet where the loss is calcu…”
MethodBacktracking regression forests for accurate camera relocalization
Meng, L., Chen, J., Tung, F., Little, J. J., Valentin, J., & de Silva, C. W. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206611 · Cited by 59 · see citing works
“Bayesian PoseNet [7] applies an uncertainty framework to the PoseNet by averaging Monte Carlo dropout samples from the posterior Bernoulli distribution of the Bayesian ConvNet’s weights.”
BackgroundMethodInfluentialLearning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images
Zhang, M., Hu, X., Zhao, L., Lv, Y., Luo, M., & Pang, S. (2017) · Remote Sensing · doi:10.3390/rs9050500 · Cited by 57 · see citing works
3D Human Pose Estimation on a Configurable Bed from a Pressure Image
Clever, H., Kapusta, A. S., Park, D., Erickson, Z., Chitalia, Y., & Kemp, C. C. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593545 · Cited by 56 · see citing works
“Monte Carlo dropout has been applied to measure uncertainty for camera relocalization [24] and semantic segmentation [25].”
Uncertainty-aware organ classification for surgical data science applications in laparoscopy
Moccia, S., Wirkert, S. J., Kenngott, H., Vemuri, A. S., Apitz, M., Mayer, B., De Momi, E., Mattos, L. S., & Maier-Hein, L. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2813015 · Cited by 56 · see citing works
“, [41], [42]), among the most recently proposed solutions, variational Bayes by Backpropagation [43], [44] is is drawing attention of the deep learning community.”
BackgroundViewFormer: NeRF-Free Neural Rendering from Few Images Using Transformers
Kulhánek, J., Derner, E., Sattler, T., & Babuška, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19784-0_12 · Cited by 55 · see citing works
Vision-Based High-Speed Driving With a Deep Dynamic Observer
Drews, P., Williams, G., Goldfain, B., Theodorou, E. A., & Rehg, J. M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2896449 · Cited by 54 · see citing works
Shen, Y., Pokharel, R., Nizolek, T. J., Kumar, A., & Lookman, T. (2019) · Acta Materialia · doi:10.1016/j.actamat.2019.03.026 · Cited by 55 · see citing works
An Improved Belief Entropy in Evidence Theory
Yan, H., & Deng, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2982579 · Cited by 55 · see citing works
Prior Guided Dropout for Robust Visual Localization in Dynamic Environments
Huang, Z., Xu, Y., Shi, J., Zhou, X., Bao, H., & Zhang, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00288 · Cited by 54 · see citing works
“[26] introduced a dropout procedure into PoseNet to measure the model’s uncertainty.”
BackgroundMethodAggressive Deep Driving: Combining Convolutional Neural Networks and Model Predictive Control
Drews, P., Williams, G., Goldfain, B., Theodorou, E. A., & Rehg, J. M. (2017) · Conference on Robot Learning · Cited by 52 · see citing works
Is Geometry Enough for Matching in Visual Localization?
Zhou, Q., Agostinho, S., Ošep, A., & Leal-Taixé, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20080-9_24 · Cited by 53 · see citing works
A critical analysis of image-based camera pose estimation techniques
Xu, M., Wang, Y., Xu, B., Zhang, J., Ren, J., Huang, Z., Poslad, S., & Xu, P. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127125 · Cited by 56 · see citing works
Li, J., Wang, C., Kang, X., & Zhao, Q. (2019) · International Journal of Digital Earth · doi:10.1080/17538947.2018.1564379 · Cited by 51 · see citing works
A Survey on Deep Learning Methods for Robot VisionPreprint
Ruíz-del-Solar, J., Loncomilla, P., & Soto, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.10862 · Cited by 67 · see citing works
“, 2016 [123] Bayesian PoseNet (Modified GoogLeNet) with pose regression, use dropout for estimating uncertainty.”
BackgroundDeep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation
Deng, H., Bui, M., Navab, N., Guibas, L. J., Ilić, S., & Birdal, T. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01612-w · Cited by 50 · see citing works
“…a range of solutions providing multiple pose hypotheses (Manhardt et al., 2019), solutions that can associate uncertainties to their predictions (Kendall & Cipolla, 2016; Manhardt et al., 2019) or even solutions in the form of full probability distributions (Birdal et al., 2018; Arun Srivatsan…”
BackgroundMethodInfluentialCoordiNet: uncertainty-aware pose regressor for reliable vehicle localization
Moreau, A., Piasco, N., Tsishkou, D., Stanciulescu, B., & Fortelle, A. D. L. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00191 · Cited by 50 · see citing works
“They generate multiple hypothesis for each single image at inference time, and then compute mean and variance to estimate pose and uncertainty [10, 9].”
BackgroundMethodLearning Camera Localization via Dense Scene Matching
Tang, S., Tang, C., Huang, R., Zhu, S., & Tan, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00187 · Cited by 49 · see citing works
“The prestigious PoseNet [23] and its varients [23, 22, 5, 21] regress the 6-DoF absolute poses directly from RGB images.”
BackgroundMethodGraph-Based Thermal–Inertial SLAM With Probabilistic Neural Networks
Saputra, M. R. U., Lu, C. X., de Gusmão, P. P. B., Wang, B., Markham, A., & Trigoni, N. (2021) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3120036 · Cited by 49 · see citing works
“This assumption has been used in [29], [56] as well.”
MethodGeometric Consistency for Self-Supervised End-to-End Visual Odometry
Iyer, G., Jatavallabhula, K. M., Gupta, G., Krishna, K. M., & Paull, L. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00064 · Cited by 49 · see citing works
“[11] use dropout [20] as a means of bayesian approximation to recover covariance from relocalization estimates from a trained CNN.”
MethodVisual Localization by Learning Objects-Of-Interest Dense Match Regression
Weinzaepfel, P., Csurka, G., Cabon, Y., & Humenberger, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00578 · Cited by 48 · see citing works
State-of-the-art in visual geo-localization
Brejcha, J., & Čadík, M. (2017) · Pattern Analysis and Applications · doi:10.1007/s10044-017-0611-1 · Cited by 46 · see citing works
“The second (parametric) option is to train classifiers or regressors, given the geotagged database of images as a train set, to directly predict the geo-coordinates of the query image [44, 90].”
MethodBeyond Controlled Environments: 3D Camera Re-localization in Changing Indoor Scenes
Wald, J., Sattler, T., Golodetz, S., Cavallari, T., & Tombari, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_28 · Cited by 46 · see citing works
“Direct pose regression methods, which aim to directly regress a pose from the query image, are often based on pose regression networks [1, 49–51,68,107], although decision forest [47], GAN [20] and LSTM [29, 103] variants also exist.”
MethodConsiderations for a more ethical approach to data in AI: On data representation and infrastructure
Baird, A., & Schuller, B. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00025 · Cited by 46 · see citing works
“Given the “black-box” nature of deep learning, there have been numerous approaches to quantifying confidence (Kendall and Cipolla, 2016; Keren et al., 2018).”
Background
- Authors
- Kendall, A., & Cipolla, R.
- Affiliations
- University of Cambridge (GB)
- ORCID
- Cipolla, Roberto
- Venue
- 2016 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2016-05
- Pages
- 4762-4769
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2279895976
- Cited by
- 548
- References
- 30
- Citations by year
- 2026: 20 · 2025: 32 · 2024: 44 · 2023: 57 · 2022: 52 · 2021: 68 · 2020: 90 · 2019: 82 · 2018: 57 · 2017: 39 · 2016: 6 · 2015: 1
- FWCI
- 107.26
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- camera relocalization, Bayesian convolutional neural network, uncertainty estimation, 6D pose estimation, real-time localization, out-of-distribution detection, indoor and outdoor scenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
