The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Single Shot MC Dropout Approximation
Open accessAbstract
Deep neural networks (DNNs) are known for their high prediction performance, especially in perceptual tasks such as object recognition or autonomous driving. Still, DNNs are prone to yield unreliable predictions when encountering completely new situations without indicating their uncertainty. Bayesian variants of DNNs (BDNNs), such as MC dropout BDNNs, do provide uncertainty measures. However, BDNNs are slow during test time because they rely on a sampling approach. Here we present a single shot MC dropout approximation that preserves the advantages of BDNNs without being slower than a DNN. Our approach is to analytically approximate for each layer in a fully connected network the expected value and the variance of the MC dropout signal. We evaluate our approach on different benchmark datasets and a simulated toy example. We demonstrate that our single shot MC dropout approximation resembles the point estimate and the uncertainty estimate of the predictive distribution that is achieved with an MC approach, while being fast enough for real-time deployments of BDNNs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Several studies used MC dropout [37] to estimate UQ.”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Kwon, Y., Won, J.-H., Kim, B. J., & Paik, M. C. (2020) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2019.106816 · Cited by 424
Research works printed but not linked: 21
- Adachi, J. Estimating and factoring the dropout induced distribution with gaussian mixture model. In International Conference on Artificial Neural Networks , pp. 775-792. Springer, 2019.
- Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., and Zieba, K. End to End Learning for Self-Driving Cars. apr 2016. URL http://arxiv.org/abs/1604.07316 . · arXiv
- Der Kiureghian, A. and Ditlevsen, O. Aleatory or epistemic? Does it matter? Structural safety , 31(2):105-112, 2009.
- Dorfman, R. A. A note on the! d-method for finding variance formulae. Biometric Bulletin , 1938.
- D¨ urr, O., Murina, E., Siegismund, D., Tolkachev, V., Steigele, S., and Sick, B. Know When You Don't Know: A Robust Deep Learning Approach in the Presence of Unknown Phenotypes. Assay and drug development technologies , 16(6):343-349, 2018.
- Feng, D., Rosenbaum, L., and Dietmayer, K. Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network for Lidar 3D Vehicle Detection. IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC , 2018-Novem:3266-3273, 2018. doi: 10.1109/ITSC.2018.8569814. · DOI
- Frey, B. J. and Hinton, G. E. Variational learning in nonlinear gaussian belief networks. Neural Computation , 11(1):193-213, 1999. ISSN 08997667. doi: 10.1162/ 089976699300016872. URL https://ieeexplore. ieee.org/abstract/document/6790581/ .
- Goodman, L. A. On the Exact Variance of Products. Journal of the American Statistical Association , 55 (292):708-713, may 1960. ISSN 01621459. doi: 10.2307/2281592. URL http://www.jstor.org/ stable/2281592http://www.cs.cmu.edu/ $\sim$cga/var/2281592.pdf . · DOI
- 362-386, oct 2019. doi: 10.1002/rob.21918. URL http://arxiv.org/abs/1910.07738http: //dx.doi.org/10.1002/rob.21918 . http://arxiv.org/abs/1910.07738%20http://dx.doi.org/10.1002/rob.21918 · DOI · Paper page
- Harakeh, A., Smart, M., and Waslander, S. L. BayesOD: A Bayesian Approach for Uncertainty Estimation in Deep Object Detectors. 2019. URL http://arxiv.org/ abs/1903.03838 . · arXiv
- Jin, J. Robust Convolutional Neural Networks under Adversarial Noise. CoRR , abs/1511.0, 2015. URL http: //arxiv.org/abs/1511.06306 .
- Kendall, A. and Gal, Y. What uncertainties do we need in Bayesian deep learning for computer vision? Technical report, 2017. URL http://papers.nips.cc/paper/ 7141-what-uncertainties-do-we-need .
- Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in neural information processing systems , pp. 6402-6413, 2017.
- MacKay, D. J. C. A Practical Bayesian Framework for Backpropagation Networks. Neural Computation , 4(3): 448-472, may 1992. ISSN 0899-7667. doi: 10.1162/ neco.1992.4.3.448.
- McAllister, R., Gal, Y., Kendall, A., Van Der Wilk, M., Shah, A., Cipolla, R., and Weller, A. Concrete problems for autonomous vehicle safety: Advantages of Bayesian deep learning. In IJCAI International Joint Conference on Artificial Intelligence , 2017. ISBN 9780999241103. doi: 10.24963/ijcai.2017/661. · DOI
- Michelmore, R., Kwiatkowska, M., and Gal, Y. Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control. nov 2018. URL http://arxiv. org/abs/1811.06817 .
- Miller, D., Nicholson, L., Dayoub, F., and Sunderhauf, N. Dropout Sampling for Robust Object Detection in OpenSet Conditions. In Proceedings - IEEE International Conference on Robotics and Automation , 2018. ISBN 9781538630815. doi: 10.1109/ICRA.2018.8460700. · DOI
- Pearce, T., Leibfried, F., Brintrup, A., Zaki, M., and Neely, A. Uncertainty in neural networks: Approximately bayesian ensembling. In The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020 , 2020.
- Ryu, S., Kwon, Y., and Kim, W. Y. A Bayesian graph convolutional network for reliable prediction of molecular properties with uncertainty quantification. Chemical Science , 10(36):8438-8446, 2019.
- Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research , 2014. ISSN 15337928.
- S¨ underhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M., and Corke, P. The limits and potentials of deep learning for robotics. International Journal of Robotics Research , 2018. ISSN 17413176. doi: 10.1177/0278364918770733. · DOI · Paper page
Cited by
Sorted by citations
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
“Several studies used MC dropout [32] to estimate UQ.”
Method
Metadata
- Authors
- Brach, K., Sick, B., & Dürr, O.
- Affiliations
- ZHAW Zurich University of Applied Sciences (CH)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2007.03293
- OpenAlex
- W3038544204
- Cited by
- 21
- References
- 2
- Citations by year
- 2025: 1 · 2024: 9 · 2023: 4 · 2022: 3 · 2021: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- University publication server of the HTWG Konstanz (Hochschule Konstanz University of Applied Sciences) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- Monte Carlo dropout, Bayesian neural networks, uncertainty estimation, approximate analytical solutions, variance estimation, fully connected networks, real-time deployment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
