Scalable Uncertainty for Computer Vision With Functional Variational Inference
Open accessAbstract
As Deep Learning continues to yield successful applications in Computer Vision, the ability to quantify all forms of uncertainty is a paramount requirement for its safe and reliable deployment in the real-world. In this work, we leverage the formulation of variational inference in function space, where we associate Gaussian Processes (GPs) to both Bayesian CNN priors and variational family. Since GPs are fully determined by their mean and covariance functions, we are able to obtain predictive uncertainty estimates at the cost of a single forward pass through any chosen CNN architecture and for any supervised learning task. By leveraging the structure of the induced covariance matrices, we propose numerically efficient algorithms which enable fast training in the context of high-dimensional tasks such as depth estimation and semantic segmentation. Additionally, we provide sufficient conditions for constructing regression loss functions whose probabilistic counterparts are compatible with aleatoric uncertainty quantification.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In 2017, Jungo et al. [659] used CNN with MC dropout for segmentation of brain tumor on 1 dataset.
“Jungo al. [659] | et | 2017 | CNN | Segmentation | Brain tumor | 1 | MC dropout”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [659]
Section 6.1, Semantic Segmentation (page unavailable), says this paper studies segmentation on the CamVid dataset and compares its method with MC dropout. It does not describe the cited work as a 2017 Jungo et al. brain-tum segmentation study; its stated segmentation setting is different. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
Spatial Interaction and the Statistical Analysis of Lattice Systems
Besag, J. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00999.x · Cited by 6,067 · see citing works
Gaussian Processes in Machine Learning
Rasmussen, C. E. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28650-9_4 · Cited by 5,727 · see citing works
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276 · see citing works
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
Cipolla, R., Gal, Y., & Kendall, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00781 · Cited by 3,657 · see citing works
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · see citing works
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · see citing works
Deeper Depth Prediction with Fully Convolutional Residual Networks
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., & Navab, N. (2016) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.32 · Cited by 2,061 · see citing works
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,802 · see citing works
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
Jégou, S., Drozdzal, M., Vázquez, D., Romero, A., & Bengio, Y. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.156 · Cited by 1,762 · see citing works
Make3D: Learning 3D Scene Structure from a Single Still Image
Saxena, A., Sun, M., & Ng, A. Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.132 · Cited by 1,738 · see citing works
Semantic object classes in video: A high-definition ground truth database
Brostow, G., Fauqueur, J., & Cipolla, R. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.04.005 · Cited by 1,400 · see citing works
Weight Uncertainty in Neural Networks
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1505.05424 · Cited by 1,342 · see citing works
Variational Learning of Inducing Variables in Sparse Gaussian Processes
Titsias, M. K. (2009) · Cited by 1,083 · see citing works
Practical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079 · see citing works
Kendall, A., Badrinarayanan, V., & Cipolla, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.57 · Cited by 784 · see citing works
Kampffmeyer, M., Salberg, A.-B., & Jenssen, R. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.90 · Cited by 618 · see citing works
Gaussian Processes for Big DataPreprint
Hensman, J. J., Fusi, N., & Lawrence, N. D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.6835 · Cited by 561 · see citing works
Accurate Uncertainties for Deep Learning Using Calibrated RegressionPreprint
Kuleshov, V., Fenner, N., & Ermon, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00263 · Cited by 455 · see citing works
Deep Bayesian Active Learning with Image Data
Ghahramani, Z., Gal, Y., & Islam, R. (2017) · PMLR · doi:10.17863/cam.11070 · Cited by 392 · see citing works
CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM
Bloesch, M., Czarnowski, J., Clark, R., Leutenegger, S., & Davison, A. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00271 · Cited by 384 · see citing works
Self-citationKendall, A., Badrinarayanan, V., & Cipolla, R. (2015) · arXiv · doi:10.48550/arxiv.1511.02680 · Cited by 292 · see citing works
Multiplicative Normalizing Flows for Variational Bayesian Neural NetworksPreprint
Louizos, C., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01961 · Cited by 254 · see citing works
VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Clark, R., Wang, S., Markham, A., Trigoni, N., & Wen, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.284 · Cited by 249 · see citing works
Self-citationLong-Term On-board Prediction of People in Traffic Scenes Under Uncertainty
Bhattacharyya, A., Fritz, M., & Schiele, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00441 · Cited by 249 · see citing works
End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks
Wang, S., Clark, R., Wen, H.-K., & Trigoni, N. (2018) · Int. J. Robotics Res. · doi:10.1177/0278364917734298 · Cited by 224 · see citing works
Self-citationBayesian Deep Convolutional Networks with Many Channels are Gaussian ProcessesPreprint
Novak, R., Xiao, L., Lee, J., Bahri, Y., Yang, G., Hron, J., Abolafia, D. A., Pennington, J., & Sohl‐Dickstein, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05148 · Cited by 210 · see citing works
Evaluating Bayesian Deep Learning Methods for Semantic SegmentationPreprint
Mukhoti, J., & Gal, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.12709 · Cited by 194 · see citing works
DeepFactors: Real-Time Probabilistic Dense Monocular SLAM
Czarnowski, J., Laidlow, T., Clark, R., & Davison, A. J. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2965415 · Cited by 188 · see citing works
Self-citationDeep Convolutional Networks as shallow Gaussian ProcessesPreprint
Garriga-Alonso, A., Aitchison, L., & Rasmussen, C. E. (2018) · arXiv (Cornell University) · doi:10.17863/cam.42340 · Cited by 176 · see citing works
Yang, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04760 · Cited by 157 · see citing works
Neural RGB®D Sensing: Depth and Uncertainty From a Video Camera
Liu, C., Gu, J., Kim, K., Narasimhan, S. G., & Kautz, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01124 · Cited by 140 · see citing works
Sampling-Free Epistemic Uncertainty Estimation Using Approximated Variance Propagation
Postels, J., Ferroni, F., Coskun, H., Navab, N., & Tombari, F. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00302 · Cited by 140 · see citing works
Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors
Louizos, C., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1603.04733 · Cited by 138 · see citing works
Efficient Uncertainty Estimation for Semantic Segmentation in Videos
Huang, P., Hsu, W., Chiu, C.-Y., Wu, T., & Sun, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_32 · Cited by 123 · see citing works
Functional Variational Bayesian Neural NetworksPreprint
Sun, S., Zhang, G., Shi, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.05779 · Cited by 116 · see citing works
VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem
Clark, R., Wang, S., Wen, H., Markham, A., & Trigoni, N. (2017) · Warwick Research Archive Portal (University of Warwick) · doi:10.5555/3298023.3298149 · Cited by 102 · see citing works
Self-citationDeep Convolutional Networks as shallow Gaussian ProcessesPreprint
Garriga-Alonso, A., Rasmussen, C. E., & Aitchison, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.05587 · Cited by 98 · see citing works
Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive PriorsPreprint
Hafner, D., Tran, D., Irpan, A., et al. (2018) · arXiv (Cornell University) · Cited by 53 · see citing works
Calculating Determinants of Block MatricesPreprint
Powell, P. D. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.4379 · Cited by 51 · see citing works
A Spectral Approach to Gradient Estimation for Implicit DistributionsPreprint
Shi, J., Sun, S., & Zhu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02925 · Cited by 43 · see citing works
Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
Yang, G. (2019) · neural information processing systems · Cited by 37 · see citing works
The adaptive BerHu penalty in robust regression
Lambert‐Lacroix, S., & Zwald, L. (2016) · Journal of nonparametric statistics · doi:10.1080/10485252.2016.1190359 · Cited by 32 · see citing works
Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic FunctionsPreprint
Pearce, T., Tsuchida, R., Zaki, M., Brintrup, A. M., & Neely, A. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06076 · Cited by 27 · see citing works
Variational Implicit Processes
Ma, C., Li, Y., & Hernández-Lobato, J. M. (2019) · Apollo (University of Cambridge) · doi:10.17863/cam.42186 · Cited by 16 · see citing works
Towards the Probabilistic Fusion of Learned Priors into Standard Pipelines for 3D Reconstruction
Laidlow, T., Czarnowski, J., Nicastro, A., Clark, R., & Leutenegger, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197001 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Mohammad Emtiyaz E Khan, Alexander Immer, Ehsan Abedi, and Maciej Korzepa. Approximate in- ference turns deep networks into gaussian processes. In Advances in Neural Information Processing Sys- tems, pages 3088-3098, 2019. 6
- Jiaxin Shi, Mohammad Emtiyaz Khan, and Jun Zhu. Scalable training of inference networks for gaussian- process models. In International Conference on Ma- chine Learning, pages 5758-5768, 2019. 2, 3, 6
- Ziyu Wang, Tongzheng Ren, Jun Zhu, and Bo Zhang. Function space particle optimization for bayesian neu- ral networks. In International Conference on Learning Representations, 2019. 6
Cited by
Sorted by citations
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
- Authors
- Carvalho, E. D. C., Clark, R., Nicastro, A., & Kelly, P. H. J.
- Affiliations
- Imperial College London (GB)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 12000-12010
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3010129597
- Cited by
- 14
- References
- 47
- Citations by year
- 2026: 1 · 2024: 2 · 2023: 2 · 2022: 4 · 2021: 5
- FWCI
- 1.05
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/P010040/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- uncertainty quantification, Gaussian processes, Bayesian convolutional neural network, aleatoric uncertainty, depth estimation, semantic segmentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
