The DOI matches its Crossref record.
Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference
Open accessAbstract
The idea to distinguish and quantify two important types of uncertainty, often referred to as aleatoric and epistemic, has received increasing attention in machine learning research in the last couple of years. In this paper, we consider ensemble-based approaches to uncertainty quantification. Distinguishing between different types of uncertainty-aware learning algorithms, we specifically focus on Bayesian methods and approaches based on so-called credal sets, which naturally suggest themselves from an ensemble learning point of view. For both approaches, we address the question of how to quantify aleatoric and epistemic uncertainty. The effectiveness of corresponding measures is evaluated and compared in an empirical study on classification with a reject option.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Second-order uncertainty prediction, a distribution of distributions, has recently been treated using inference based on Bayesian methods and credal sets.
“This type of prediction, a distribution of distributions, also called second-order uncertainty (Hüllermeier et al., 2022;Bengs et al., 2022), has been recently treated using inference based on Bayesian methods and credal sets, which are convex sets of probability distributions (Shaker & Hüllermeier, 2021).”
From Classifier calibration: a survey on how to assess and improve predicted class probabilities (2023) · cites this paper as Shaker & Hüllermeier, 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 20
- J. Abellan and J. Klir. Additivity of uncertainty measures on credal sets. International Journal of General Systems, 34 (6):691-713, 2005.
- J. Abellan and S. Moral. A non-specificity measure for convex sets of probability distributions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 8:357-367, 2000.
- J. Abellan, J. Klir, and S. Moral. Disaggregated total uncertainty measure for credal sets. International Journal of General Systems, 35(1), 2006.
- J.M. Bernardo. An introduction to the imprecise Dirichlet model for multinomial data. International Journal of Approximate Reasoning, 39(2-3):123-150, 2005.
- S. Depeweg, J.M. Hernandez-Lobato, F. Doshi-Velez, and S. Udluft. Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In Proc. ICML, Stockholm, Sweden, 2018.
- Y. Gal and Z. Ghahramani. Bayesian convolutional neural networks with Bernoulli approximate variational inference. In Proc. of the ICLR Workshop Track, 2016.
- K. Hara, D. Saitoh, and H. Shouno. Analysis of dropout learning regarded as ensemble learning. In Proc. ICANN 2016, Part II, number 9887 in LNCS, pages 1-8. 2016.
- R.V.L. Hartley. Transmission of information. Bell Syst. Tech. Journal, 7(3):535-563, 1928.
- S.C. Hora. Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management. Reliability Engineering and System Safety, 54(2-3):217-223, 1996.
- J. Hühn and E. Hüllermeier. FR3: A fuzzy rule learner for inducing reliable classifiers. IEEE Transactions on Fuzzy Systems, 17(1):138-149, 2009.
- A. Kendall and Y. Gal. What uncertainties do we need in Bayesian deep learning for computer vision? In Proc. NIPS, pages 5574-5584, 2017.
- G.J. Klir and M. Mariano. On the uniqueness of possibilistic measure of uncertainty and information. Fuzzy Sets and Systems, 24(2):197-219, 1987.
- A.N. Kolmogorov. Three approaches to the quantitative definition of information. Problems Inform. Trans., 1(1):1-7, 1965.
- B. Lakshminarayanan, A. Pritzel, C. and Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Proc. NeurIPS, 2017.
- I. Levi. On indeterminate probabilities. Journal of Philosophy, 71:391-418, 1974.
- I. Levi. The Enterprise of Knowledge. MIT Press, Cambridge, 1980.
- A. Mobiny, H.V. Nguyen, S. Moulik, N. Garg, and C.C. Wu. DropConnect is effective in modeling uncertainty of Bayesian networks. CoRR, abs/1906.04569, 2017. · arXiv
- N. Papernot and P. McDaniel. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. CoRR, abs/1803.04765v1, 2018. · arXiv
- G. Shafer. A Mathematical Theory of Evidence. Princeton University Press, 1976.
- R.R. Yager. Entropy and specificity in a mathematical theory of evidence. International Journal of General Systems, 9: 249-260, 1983.
Cited by
Sorted by citations
Classifier calibration: a survey on how to assess and improve predicted class probabilities
Silva Filho, T., Song, H., Perello-Nieto, M., Santos-Rodriguez, R., Kull, M., & Flach, P. (2023) · Machine Learning · doi:10.1007/s10994-023-06336-7 · Cited by 228
Metadata
- Authors
- Shaker, M., & Hüllermeier, E.
- Affiliations
- Paderborn University (DE)
- Ludwig-Maximilians-Universität München (DE)
- Venue
- Proceedings - 31. Workshop Computational Intelligence : Berlin, 25. - 26. November 2021
- Publisher
- KIT Scientific Publishing
- Published
- 2021-11-15
- Pages
- 63-77
- OpenAlex
- W3186967849
- Cited by
- 9
- References
- 0
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 1 · 2023: 1 · 2022: 2
- FWCI
- 0.46
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-sa
- Locations
- Workshop Computational Intelligence (publishedVersion, cc-by-sa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Bundesministerium für Bildung und Forschung (01IS19080 A)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Keywords
- uncertainty quantification, ensemble learning, aleatoric uncertainty, epistemic uncertainty, Bayesian inference, credal sets, uncertainty-aware learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
