Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
Open accessAbstract
The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this includes the importance of distinguishing between (at least) two different types of uncertainty, often referred to as aleatoric and epistemic . In this paper, we provide an introduction to the topic of uncertainty in machine learning as well as an overview of attempts so far at handling uncertainty in general and formalizing this distinction in particular.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Predictive multiplicity is high when all predictions are close to τ but some lie above and others below (high aleatoric uncertainty), or when predictions are widely spread with some far above and others far below τ (high epistemic uncertainty).
“Predictive multiplicity, and thus the prospects for successful epistemic contestability, is high in two scenarios: (1) when all predictions are close to τ but some lie above and others below, corresponding to high aleatoric uncertainty; and (2) when predictions are widely spread, with some far above and others far below τ , corresponding to high epistemic uncertainty (Hüllermeier and Waegeman 2021”
From Explainable AI Isn't Enough! Rethinking Algorithmic Contestability (2026) · cites this paper as Hüllermeier and Waegeman 2021
Section 4.7.2 (page unavailable) links predictions near the 1/2 class boundary to aleatoric uncertainty and strongly opposed predictions from plausible hypotheses to epistemic uncertainty. It also says full aleatoric uncertainty involves agreement at 1/2, not predictions falling on both sides of a threshold. The paper does not establish the claimed two-scenario rule for predictive multiplicity or its implications for epistemic contestability. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1× · see citing works
“Especially important in this field are tree-based methods such as random forests (Breiman, 2001; Kruppa et al., 2014).”
MethodEfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 1× · see citing works
“Figure 1: Predictions by EfficientNet (Tan and Le, 2019) on test images from ImageNet: For the left image, the neural network predicts “typewriter keyboard” with certainty 83.”
Method(2016) · Cited by 11,679 · see citing works
A Mathematical Theory of Evidence
Shafer, G. R. (1976) · Princeton University Press eBooks · doi:10.1515/9780691214696 · Cited by 9,092 · see citing works
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601 · cited 1× · see citing works
“Some more recent methods in this area are isolation forests (Liu et al., 2009), auto-encoders (Goodfellow et al.”
MethodSimple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · cited 1× · see citing works
“Based on this idea, Lakshminarayanan et al. (2017) propose a simple ensemble approach as an alternative to Bayesian NNs, which is easy to implement, readily parallelizable, and requires little hyperparameter tuning.”
MethodAn Introduction to Variational Methods for Graphical Models
Jordan, M. I., Zoubin Ghahramani, A., Jaakkola, T., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,665 · see citing works
Support Vector Data Description
Tax, D. M. J., & Duin, R. P. W. (2003) · Machine Learning · doi:10.1023/b:mach.0000008084.60811.49 · Cited by 3,553 · see citing works
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · cited 2× · see citing works
“To capture this kind of epistemic uncertainty, Bayesian neural networks (BNNs) have been proposed as a Bayesian extension of deep neural networks (Denker and LeCun, 1991; Kay, 1992; Neal, 2012).”
MethodStatistical Reasoning with Imprecise Probabilities
Walley, P. (1991) · Springer eBooks · doi:10.1007/978-1-4899-3472-7 · Cited by 2,678 · cited 3× · see citing works
“…a single (probability) distribution is not enough for representing uncertain knowledge is quite prominent in the literature, and many generalized theories of uncertainty can be considered as a combination of that kind (Dubois and Prade, 1988; Walley, 1991; Shafer, 1976; Smets and Kennes, 1994).”
BackgroundAleatory or epistemic? Does it matter?
Kiureghian, A. D., & Ditlevsen, O. (2009) · Structural Safety · doi:10.1016/j.strusafe.2008.06.020 · Cited by 2,454 · see citing works
An invariant form for the prior probability in estimation problems
Jeffreys, H. (1946) · Proceedings of the Royal Society of London A Mathematical and Physical Sciences · doi:10.1098/rspa.1946.0056 · Cited by 2,347 · cited 3× · see citing works
“See work on “non-informative” priors, however (Jeffreys, 1946; Bernardo, 1979). dominated probability distributions: P ..= { P ∈ P”
BackgroundKlement, E. P., Mesiar, R., & Pap, E. (2000) · Trends in logic · doi:10.1007/978-94-015-9540-7 · Cited by 2,246 · see citing works
Smets, P., & Kennes, R. (1994) · Artificial Intelligence · doi:10.1016/0004-3702(94)90026-4 · Cited by 2,234 · see citing works
Vapnik, V. N. 1. (1998) · Cited by 2,163 · see citing works
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968 · cited 1× · see citing works
“In a classification problem in machine learning, for example, not all possible classes might be known beforehand, or new classes may emerge in the course of time (Hendrycks and Gimpel 2017; Liang et al. 2018; DeVries and Taylor 2018).”
BackgroundHartley, R. V. L. (1928) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1928.tb01236.x · Cited by 1,844 · cited 1× · see citing works
“The standard uncertainty measure in classical possibility theory (where uncertain information is simply represented in the form of subsets A ⊆ Y of possible alternatives) is the Hartley measure12 (Hartley, 1928) H(A) = log(|A”
BackgroundThe Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,652 · cited 1× · see citing works
“This is closely connected to the insight that learning from data necessarily requires an inductive bias, and learning without assumptions is not possible (Wolpert, 1996).”
BackgroundGAUSSIAN PROCESSES FOR MACHINE LEARNING
Seeger, M. (2004) · International Journal of Neural Systems · doi:10.1142/s0129065704001899 · Cited by 1,518 · cited 2× · see citing works
“Gaussian processes (Seeger, 2004) generalize the Bayesian approach from inference about multivariate (but finite-dimensional) random variables to inference about (infinite-dimensional) functions.”
On optimum recognition error and reject tradeoff
Chow, C.-K. (1970) · IEEE Transactions on Information Theory · doi:10.1109/tit.1970.1054406 · Cited by 1,303 · cited 1× · see citing works
“Instead, the ultimate decision can often be refused or delayed, like in classification with a reject option (Chow, 1970; Hellman, 1970), or actions can be taken that are specifically directed at reducing uncertainty, like in active learning (Aggarwal et al., 2014).”
BackgroundTransforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280 · cited 2× · see citing works
“Examples of such methods include binning (Zadrozny and Elkan, 2001), isotonic regression (Zadrozny and Elkan, 2002), logistic scaling (Platt, 1999) and improvements thereof (Kull et al.”
MethodA new metric for probability distributions
Endres, D., & Schindelin, J. E. (2003) · IEEE Transactions on Information Theory · doi:10.1109/tit.2003.813506 · Cited by 1,212 · see citing works
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial AttacksPreprint
Lee, K., Lee, K., Lee, H., & Shin, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03888 · Cited by 1,166 · cited 1× · see citing works
“Some first proposals for dealing with this case can already be found in the literature (DeVries and Taylor, 2018; Lee et al., 2018b; Malinin and Gales, 2018; Sensoy et al., 2018).”
BackgroundAlgorithmic Learning in a Random World
(2005) · doi:10.1007/b106715 · Cited by 1,119 · see citing works
Enhancing The Reliability of Out-of-distribution Image Detection in Neural NetworksPreprint
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 1,117 · cited 1× · see citing works
“In a classification problem in machine learning, for example, not all possible classes might be known beforehand, or new classes may emerge in the course of time (Hendrycks and Gimpel, 2017; Liang et al., 2018; DeVries and Taylor, 2018).”
BackgroundReference Posterior Distributions for Bayesian Inference
Bernardo, J. M. (1979) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1979.tb01066.x · Cited by 1,083 · cited 2× · see citing works
“See work on “non-informative” priors, however (Jeffreys, 1946; Bernardo, 1979). dominated probability distributions: P ..= { P ∈ P”
BackgroundPractical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079 · see citing works
Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational InferencePreprint
Gal, Y., & Ghahramani, Z. (2015) · arXiv.org · doi:10.48550/arxiv.1506.02158 · Cited by 839 · see citing works
Evidential Deep Learning to Quantify Classification Uncertainty
Sensoy, M., Kaplan, L., & Kandemir, M. (2018) · arXiv · doi:10.48550/arxiv.1806.01768 · Cited by 694 · cited 2× · see citing works
“Some first proposals for dealing with this case can already be found in the literature (DeVries and Taylor, 2018; Lee et al., 2018b; Malinin and Gales, 2018; Sensoy et al., 2018).”
BackgroundPukelsheim, F. (2006) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898719109 · Cited by 687 · see citing works
(2001) · Cited by 683 · see citing works
A tutorial on conformal prediction
Shafer, G., & Vovk, V. (2007) · arXiv · doi:10.48550/arxiv.0706.3188 · Cited by 655 · cited 1× · see citing works
“Conformal prediction (Vovk et al., 2003; Shafer and Vovk, 2008; Balasubramanian et al., 2014) is a framework for reliable prediction that is rooted in classical frequentist statistics, more specifically in hypothesis testing.”
BackgroundOne-class classification: taxonomy of study and review of techniques
Khan, S. S., & Madden, M. G. (2014) · The Knowledge Engineering Review · doi:10.1017/s026988891300043x · Cited by 605 · see citing works
Deep Neural Networks as Gaussian ProcessesPreprint
Lee, J., Bahri, Y., Novak, R., Schoenholz, S. S., Pennington, J., & Sohl‐Dickstein, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00165 · Cited by 603 · cited 1× · see citing works
“Some first proposals for dealing with this case can already be found in the literature (DeVries and Taylor, 2018; Lee et al., 2018b; Malinin and Gales, 2018; Sensoy et al., 2018).”
BackgroundScience from Fisher Information
Frieden, B. R. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511616907 · Cited by 586 · see citing works
Predictive Uncertainty Estimation via Prior NetworksPreprint
Malinin, A., & Gales, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10501 · Cited by 488 · cited 6× · see citing works
“For example, imagine that a sudden change of the distribution cannot be excluded, or a strong discrepancy between training and test data (Malinin and Gales, 2018).”
BackgroundInfluentialPossibility theory and statistical reasoning
Dubois, D. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.04.015 · Cited by 467 · cited 2× · see citing works
“Thus, Π is “maxitive” instead of additive (Shilkret, 1971; Dubois, 2006).”
BackgroundDeep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep LearningPreprint
Papernot, N., & McDaniel, P. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04765 · Cited by 460 · cited 2× · see citing works
“Moreover, such models are often lacking robustness and can easily be fooled by “adversarial examples” (Papernot and McDaniel 2018): Drastic changes of a prediction may already be provoked by minor, actually unimportant changes of an object.”
BackgroundENTROPY AND SPECIFICITY IN A MATHEMATICAL THEORY OF EVIDENCE
Yager, R. R. (1983) · International Journal of General Systems · doi:10.1080/03081078308960825 · Cited by 420 · see citing works
Learning Confidence for Out-of-Distribution Detection in Neural NetworksPreprint
DeVries, T., & Taylor, G. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04865 · Cited by 354 · cited 2× · see citing works
“Some first proposals for dealing with this case can already be found in the literature (DeVries and Taylor, 2018; Lee et al., 2018b; Malinin and Gales, 2018; Sensoy et al., 2018).”
BackgroundCozman, F. G. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00029-1 · Cited by 345 · cited 1× · see citing works
“In the literature on imprecise probability, these are also called credal sets (Cozman, 2000; Zaffalon, 2002b).”
BackgroundMaxitive measure and integration
Shilkret, N. (1971) · Indagationes Mathematicae (Proceedings) · doi:10.1016/s1385-7258(71)80017-3 · Cited by 339 · cited 1× · see citing works
“Thus, Π is “maxitive” instead of additive (Shilkret, 1971; Dubois, 2006).”
BackgroundOn random sets and belief functions
Nguyen, H. T. (1978) · Journal of Mathematical Analysis and Applications · doi:10.1016/0022-247x(78)90161-0 · Cited by 334 · see citing works
Version spaces: a candidate elimination approach to rule learning
Mitchell, T. M. (1977) · International Joint Conference on Artificial Intelligence · Cited by 333 · see citing works
Deng, Y. (2015) · Applied Intelligence · doi:10.1007/s10489-015-0661-2 · Cited by 301 · cited 2× · see citing works
“Related to this is the “closed world” assumption (Deng, 2014), which is often violated in contemporary machine learning applications.”
BackgroundAxiomatic Characterizations of Information Measures
Csiszár, I. (2008) · Entropy · doi:10.3390/e10030261 · Cited by 281 · cited 1× · see citing works
“…a measure of total uncertainty as well as its decomposition into aleatoric and epistemic parts from basic principles in a mathematically rigorous way, i.e., to develop a kind of axiomatic basis for such a decomposition, comparable to axiomatic justifications of the entropy measure (Csiszár, 2008).”
BackgroundHora, S. C. (1996) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(96)00077-4 · Cited by 249 · cited 1× · see citing works
“…modeling, which are essentially based on capturing knowledge in terms of a single probability distribution, fail to distinguish two inherently different sources of uncertainty, which are often referred to as aleatoric and epistemic uncertainty (Hora, 1996; Der Kiureghian and Ditlevsen, 2009).”
BackgroundUncertainty and Vagueness in Knowledge Based Systems
Kruse, R., Schwecke, E., & Heinsohn, J. (1991) · Artificial intelligence · doi:10.1007/978-3-642-76702-9 · Cited by 224 · see citing works
Transforming Neural-Net Output Levels to Probability Distributions
Denker, J. S., & LeCun, Y. (1990) · Cited by 223 · see citing works
The Nearest Neighbor Classification Rule with a Reject Option
Hellman, M. E. (1970) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1970.300339 · Cited by 219 · cited 2× · see citing works
“Instead, the ultimate decision can often be refused or delayed, like in classification with a reject option (Chow, 1970; Hellman, 1970), or actions can be taken that are specifically directed at reducing uncertainty, like in active learning (Aggarwal et al., 2014).”
BackgroundOn the safety of machine learning: Cyber-physical systems, decision sciences, and data products
Varshney, K. R., & Alemzadeh, H. (2017) · Big Data · doi:10.1089/big.2016.0051 · Cited by 211 · cited 2× · see citing works
“On the more practical side, Varshney and Alemzadeh (2016) give an example of a recent accident of a self-driving car, which led to the death of the driver (for the first time after 130 million miles of testing).”
BackgroundClassification with reject option
Herbei, R., & Wegkamp, M. H. (2006) · Canadian Journal of Statistics · doi:10.1002/cjs.5550340410 · Cited by 203 · see citing works
Inductive Conformal Prediction: Theory and Application to Neural Networks
Papadopoulos, H. (2008) · InTech eBooks · doi:10.5772/6078 · Cited by 195 · cited 2× · see citing works
“Conformal prediction as outlined above realizes transductive inference, although inductive variants also exist (Papadopoulos 2008).”
BackgroundAn introduction to the imprecise Dirichlet model for multinomial data
Bernard, J. (2004) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2004.10.002 · Cited by 178 · cited 1× · see citing works
“An important example of such an approach is robust inference for categorical data with the imprecise Dirichlet model, which is an extension of inference with the Dirichlet distribution as a conjugate prior for the multinomial distribution (Bernardo, 2005).”
MethodInterpretable Adversarial Perturbation in Input Embedding Space for Text
Sato, M., Suzuki, J., Shindo, H., & Matsumoto, Y. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/601 · Cited by 170 · cited 2× · see citing works
“Figure 2: Adversarial example (right) misclassified by a machine learning model trained on textual data: Changing only a single — and apparently not very important — word (highlighted in bold font) is enough to turn the correct prediction “negative sentiment” into the incorrect prediction “positive sentiment” (Sato et…”
MethodThe limits of distribution-free conditional predictive inference
Barber, R. F., Candès, E. J., Ramdas, A., & Tibshirani, R. J. (2020) · Information and Inference A Journal of the IMA · doi:10.1093/imaiai/iaaa017 · Cited by 163 · cited 1× · see citing works
“Roughly speaking, in the case of marginal coverage, one averages over both XN+1 and YN+1 (i.e., the probability P refers to a joint measure on X × Y), while in the case of conditional coverage, XN+1 is fixed and the average in taken over YN+1 only (Barber et al., 2020).”
BackgroundA survey on learning from data streams: current and future trends
da Gama, J. M. P. (2012) · Progress in Artificial Intelligence · doi:10.1007/s13748-011-0002-6 · Cited by 162 · cited 1× · see citing works
“Likewise, non-stationarity of the data-generating process (the measure P ), including the possibility of drift or shift of the distribution, is a common assumption in learning on data streams (Gama, 2012).”
BackgroundConformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications
Balasubramanian, V. N., Ho, S.-S., & Vovk, V. (2014) · Cited by 155 · see citing works
A Semantics for Possibility Theory Based on Likelihoods
Dubois, D., Moral, S., & Prade, H. M. (1997) · Journal of Mathematical Analysis and Applications · doi:10.1006/jmaa.1997.5193 · Cited by 152 · see citing works
Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty
Senge, R., Bösner, S., Dembczyński, K., Haasenritter, J., Hirsch, O., Donner‐Banzhoff, N., & Hüllermeier, E. (2013) · Information Sciences · doi:10.1016/j.ins.2013.07.030 · Cited by 146 · see citing works
Self-citationZaffalon, M. (2002) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(01)00201-4 · Cited by 144 · cited 4× · see citing works
“(24) In the Näıve Credal Classifier, the θk and θij ,j,k are specified in terms of (local) credal sets11, i.e., there is a class of probability distributions Q on Y playing the role of candidate 11These could be derived from empirical data, for example, by doing generalized Bayesian inference with the imprecise Diric…”
BackgroundMethodInfluentialUnifying practical uncertainty representations – I: Generalized p-boxes
Destercke, S., Dubois, D., & Chojnacki, E. (2008) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2008.07.003 · Cited by 134 · see citing works
Using random forest for reliable classification and cost-sensitive learning for medical diagnosis
Yang, F., Wang, H., Mi, H., Lin, C., & Cai, W. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-s1-s22 · Cited by 130 · cited 2× · see citing works
“…of uncertainty is desirable and should be considered as a key feature of any machine learning method, all the more in safety-critical application domains such as medicine (Yang et al., 2009; Lambrou et al., 2011) or sociotechnical systems (Varshney, 2016; Varshney and Alemzadeh, 2016).”
BackgroundOptimal Uncertainty Quantification
Owhadi, H., Scovel, C., Sullivan, T. J., McKerns, M. M., & Ortíz, M. (2013) · SIAM Review · doi:10.1137/10080782x · Cited by 129 · see citing works
Representing partial ignorance
Dubois, D., Prade, H. M., & Smets, P. (1996) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/3468.487961 · Cited by 128 · see citing works
Engineering safety in machine learning
Varshney, K. R. (2016) · Information Theory and Applications Workshop (ITA) · doi:10.1109/ita.2016.7888195 · Cited by 128 · cited 2× · see citing works
“…of uncertainty is desirable and should be considered as a key feature of any machine learning method, all the more in safety-critical application domains such as medicine (Yang et al., 2009; Lambrou et al., 2011) or sociotechnical systems (Varshney, 2016; Varshney and Alemzadeh, 2016).”
BackgroundKull, M., Filho, T. M. S., & Flach, P. A. (2017) · Bristol Research (University of Bristol) · Cited by 113 · see citing works
Belief functions and statistical inference
Wasserman, L. (1990) · Canadian Journal of Statistics · doi:10.2307/3315449 · Cited by 103 · cited 1× · see citing works
“The normalized likelihood was originally introduced by Shafer (1976), and has been justified axiomatically in the context of statistical inference by Wasserman (1990).”
MethodLikelihood-based belief function: Justification and some extensions to low-quality data
Denœux, T. (2013) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2013.06.007 · Cited by 101 · cited 1× · see citing works
“Further arguments in favor of using the relative likelihood as the contour function of a (consonant) plausibility function are provided by Denoeux (2014), who shows that it can be derived from three basic principles: the likelihood principle, compatibility with Bayes’ rule, and the so-called…”
BackgroundKruppa, J., Liu, Y., Biau, G., Kohler, M., König, I. R., Malley, J. D., & Ziegler, A. (2014) · Biometrical Journal · doi:10.1002/bimj.201300068 · Cited by 97 · see citing works
On the uniqueness of possibilistic measure of uncertainty and information
Klir, G. J., & Mariano, M. (1987) · Fuzzy Sets and Systems · doi:10.1016/0165-0114(87)90090-x · Cited by 94 · see citing works
Learning valued preference structures for solving classification problems
Hüllermeier, E., & Brinker, K. (2008) · Fuzzy Sets and Systems · doi:10.1016/j.fss.2008.01.021 · Cited by 94 · see citing works
Self-citationFR3: A Fuzzy Rule Learner for Inducing Reliable Classifiers
Hühn, J., & Hüllermeier, E. (2008) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2008.2005490 · Cited by 92 · see citing works
Self-citationAggarwal, C. C., Kong, X., Gu, Q., Han, J., & Yu, P. S. (2014) · doi:10.1201/b17320-27 · Cited by 92 · see citing works
Learning Nondeterministic Classifiers
del Coz, J. D., Díez, J., & Bahamonde, A. (2009) · Journal of machine learning research · doi:10.5555/1577069.1755862 · Cited by 83 · cited 2× · see citing works
“Various methods in this direction have been proposed under different names and qualifications of predictions, such as “indeterminate” (Zaffalon, 2002a), “credal” (Corani and Zaffalon, 2008a), “non-deterministic” (Del Coz et al., 2009), and “cautious” (Yang et al., 2017b).”
BackgroundMethodEvaluating credal classifiers by utility-discounted predictive accuracy
Zaffalon, M., Corani, G., & Mauá, D. D. (2012) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2012.06.022 · Cited by 73 · see citing works
Aleatoric and Epistemic Uncertainty with Random Forests
Shaker, M., & Hüllermeier, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-44584-3_35 · Cited by 68 · see citing works
Self-citationLearning Reliable Classifiers From Small or Incomplete Data Sets: The Naive Credal Classifier 2
Corani, G., & Zaffalon, M. (2008) · Cited by 62 · see citing works
Optimization of Gaussian Process Hyperparameters using Rprop
Blum, M., & Riedmiller, M. (2013) · The European Symposium on Artificial Neural Networks · Cited by 62 · see citing works
Reliable Confidence Measures for Medical Diagnosis With Evolutionary Algorithms
Lambrou, A., Papadopoulos, H., & Gammerman, A. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2010.2091144 · Cited by 61 · see citing works
Disaggregated total uncertainty measure for credal sets
Abellán, J., Klir, G. J., & Moral, S. (2006) · International Journal of General Systems · doi:10.1080/03081070500473490 · Cited by 61 · see citing works
Prediction algorithms and confidence measures based on algorithmic randomness theory
Gammerman, A., & Vovk, V. (2002) · Theoretical Computer Science · doi:10.1016/s0304-3975(02)00100-7 · Cited by 60 · see citing works
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2005) · doi:10.21236/ada454828 · Cited by 50 · see citing works
A Tutorial on Hierarchical Classification with Applications in Bioinformatics
Freitas, A. A., & de Carvalho, A. C. P. L. F. (2007) · IGI Global eBooks · doi:10.4018/978-1-59904-271-8.ch007 · Cited by 45 · cited 1× · see citing works
“Set-valued predictions are also considered in hierarchical multi-class classification, mostly in the form of internal nodes of the hierarchy (Freitas, 2007; Rangwala and Naik, 2017; Yang et al., 2017a).”
BackgroundDeep Gamblers: Learning to Abstain with Portfolio TheoryPreprint
Ziyin, L., Wang, Z., Liang, P. P., Salakhutdinov, R., Morency, L., & Ueda, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00208 · Cited by 40 · cited 1× · see citing works
“For example, Ziyin et al. (2019) adopt this viewpoint in an optimization framework where an outlier detector and a standard classifier are jointly optimized.”
BackgroundComparing probability measures using possibility theory: A notion of relative peakedness
Dubois, D., & Hüllermeier, E. (2006) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2006.06.017 · Cited by 37 · see citing works
Self-citationA NON-SPECIFICITY MEASURE FOR CONVEX SETS OF PROBABILITY DISTRIBUTIONS
Abellán, J., & Moral, S. (2000) · International Journal of Uncertainty Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488500000253 · Cited by 37 · see citing works
Bayes-Optimal Hierarchical Multilabel Classification
Bi, W., & Kwok, J. T. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2441707 · Cited by 36 · see citing works
Epistemic Uncertainty Sampling
Nguyen, V.-L., Destercke, S., & Hüllermeier, E. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-33778-0_7 · Cited by 31 · see citing works
Self-citationCautious Deep LearningPreprint
Hechtlinger, Y., Póczos, B., & Wasserman, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09460 · Cited by 29 · cited 1× · see citing works
“Hechtlinger et al. (2019) implement this idea by fitting a generative model p(x | y) per class, and predicting null sets for data points that have a too low density for any of those generative models.”
MethodA Probabilistic Active Learning Algorithm Based on Fisher Information Ratio
Sourati, J., Akçakaya, M., Deniz Erdogmus, B., Leen, T. K., & Dy, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2743707 · Cited by 28 · see citing works
Depeweg, S., Hernández-Lobato, J. M., Doshi-Velez, F., & Udluft, S. (2017) · arXiv.org · doi:10.48550/arxiv.1710.07283 · Cited by 27 · see citing works
Measures of uncertainty in the Dempster-Shafer theory of evidence
Klir, G. J. (1994) · John Wiley & Sons, Inc. eBooks · Cited by 24 · see citing works
Reliable Multi-class Classification based on Pairwise Epistemic and Aleatoric Uncertainty
Nguyen, V.-L., Destercke, S., Masson, M.-H., & Hüllermeier, E. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/706 · Cited by 24 · cited 7× · see citing works
“Other utility functions with specific choices for g are studied in the literature on credal classification (Corani and Zaffalon, 2008b, 2009; Zaffalon et al., 2012; Yang et al., 2017b; Nguyen et al., 2018), such as”
BackgroundInfluentialSelf-citationReliability Maps: A Tool to Enhance Probability Estimates and Improve Classification Accuracy
Kull, M., & Flach, P. A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44851-9_2 · Cited by 20 · see citing works
The Costs of Indeterminacy: How to Determine Them?
Yang, G., Destercke, S., & Masson, M.-H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2607237 · Cited by 18 · see citing works
Corani, G., & Zaffalon, M. (2009) · doi:10.1145/1610555.1610560 · Cited by 16 · cited 1× · see citing works
“Other utility functions with specific choices for g are studied in the literature on credal classification (Corani and Zaffalon, 2008b, 2009; Zaffalon et al., 2012; Yang et al., 2017b; Nguyen et al., 2018), such as gδ,γ(s) ..= δ s − γ s2 , gexp(s) ..= 1− exp ( −δ s ) , glog(s) ..= log ( 1 + 1 s ) .”
BackgroundBackground Check: A General Technique to Build More Reliable and Versatile Classifiers
Perelló-Nieto, M., Filho, T. M. S., Kull, M., & Flach, P. A. (2016) · Proceedings · doi:10.1109/icdm.2016.0150 · Cited by 14 · see citing works
Reliable Confidence Predictions Using Conformal Prediction
Linusson, H., Johansson, U., Boström, H., & Löfström, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-31753-3_7 · Cited by 13 · cited 1× · see citing works
“Besides, other methods for quantifying the uncertainty of a point prediction in the context of conformal prediction have been proposed (Linusson et al., 2016).”
MethodTop-k Hierarchical Classification
Oh, S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10813 · Cited by 12 · cited 1× · see citing works
“For example, Yang et al. (2017a) evaluate various members of the uδ,γ family in a framework where hierarchies are considered for computational reasons, while Oh (2017) optimizes recall by fixing |Ŷ | as a user-defined parameter.”
BackgroundCautious classification with nested dichotomies and imprecise probabilities
Yang, G., Destercke, S., & Masson, M.-H. (2016) · Soft Computing · doi:10.1007/s00500-016-2287-7 · Cited by 11 · see citing works
Selective prediction-set models with coverage guaranteesPreprint
Feng, J., Sondhi, A., Perry, J., & Simon, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05473 · Cited by 11 · see citing works
Consistent Algorithms for Multiclass Classification with a Reject OptionPreprint
Ramaswamy, H. G., Tewari, A., & Agarwal, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04137 · Cited by 10 · cited 1× · see citing works
“Another example appears in the literature on binary or multi-class classification with reject option (Herbei and Wegkamp, 2006; Linusson et al., 2018; Ramaswamy et al., 2015).”
BackgroundClassification with Reject Option Using Conformal Prediction
Linusson, H., Johansson, U., Boström, H., & Löfström, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93034-3_8 · Cited by 10 · cited 1× · see citing works
“Another example appears in the literature on binary or multi-class classification with reject option (Herbei and Wegkamp, 2006; Linusson et al., 2018; Ramaswamy et al., 2015).”
BackgroundFlach, P. A. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_900 · Cited by 10 · see citing works
Likelihood-Based Statistical Decisions
Cattaneo, M. (2005) · Cited by 10 · see citing works
Representing credal imprecision: from sets of measures to hierarchical Bayesian models
Lassiter, D. (2019) · Philosophical Studies · doi:10.1007/s11098-019-01262-8 · Cited by 6 · cited 3× · see citing works
“Lassiter (2020) discusses the modeling of “credal imprecision” by means of sets of measures in a critical way and contrasts this approach with hierarchical probabilistic (Bayesian) models, i.e., distributions of measures (see Fig.”
MethodInfluentialVenn predictors for well-calibrated probability estimation trees
Johansson, U., Löfström, T., Sundell, H., Linusson, H., Gidenstam, A., & Boström, H. (2018) · Diva portal (Dalarna University Library) · Cited by 5 · see citing works
Active Learning by the Naive Credal Classifier
Antonucci, A., Giorgio Corani, A., & Bernaschina, S. (2012) · Cited by 5 · see citing works
Flagged first, then by citations
2 of the 111 references couldn’t be matched to a known paper.
Research works printed but not linked: 14
- Bazargami, M., & Mac-Namee, B. (2019). The elliptical basis function data descriptor network: A one- class classification approach for anomaly detection. In European conference on machine learning and knowledge discovery in databases.
- Depeweg, S., Hernandez-Lobato, J., Doshi-Velez, F., & Udluft, S. (2018). Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In Proceedings of ICML, 35th inter- national conference on machine learning, Stockholm, Sweden.
- Dubois, D., & Prade, H. (1988). Possibility theory. Berlin: Plenum Press.
- Hartley, R. (1928). Transmission of information. Bell Labs Technical Journal, 7(3), 535-563.
- Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Proceedings of NIPS, advances in neural information processing systems, pp. 5574-5584.
- Kolmogorov, A. (1965). Three approaches to the quantitative definition of information. Problems of Infor- mation Transmission, 1(1), 1-7.
- Maau, D. D., Cozman, F., Conaty, D., & de Campos, C. P. (2017). Credal sum-product networks. In PMLR: Proceedings of machine learning Research (ISIPTA 2017), vol 62, pp. 205-216.
- Matheron, G. (1975). Random sets and integral geometry. New York: Wiley.
- Mobiny, A., Nguyen, H., Moulik, S., Garg, N., & Wu, C. (2017). DropConnect is effective in modeling uncertainty of Bayesian networks. CoRR. arXiv :abs/1906.04569 .
- Neal, R. (2012). Bayesian learning for neural networks (p. 118). Berlin: Springer.
- Owhadi, H., Sullivan, T., McKerns, M., Ortiz, M., & Scovel, C. (2012). Optimal uncertainty quantifica- tion. CoRR. (arXiv :abs/1009.0679.v3).
- Platt, J. (1999). Probabilistic outputs for support vector machines and comparison to regularized likeli- hood methods. In A. Smola, P. Bartlett, B. Schoelkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers (pp. 61-74). Cambridge, MA: MIT Press.
- Rangwala, H., & Naik, A. (2017). Large scale hierarchical classification: Foundations, algorithms and applications. In The European conference on machine learning and principles and practice of knowledge discovery in databases.
- Sugeno, M. (1974). Theory of fuzzy integrals and its application. PhD thesis, Tokyo Institute of Technology.
Cited by
Sorted by citations
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
A survey of uncertainty in deep neural networks
Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10562-9 · Cited by 1,525 · see citing works
“uncertainty [57]) and model uncertainty (also systemic or epistemic uncertainty [57]).”
BackgroundTowards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
“Since these two types of uncertainties (EPISTEMIC and ALEATORIC) are highly context-dependent, changing the context may blur the difference between them or even cause one to turn into the other.”
BackgroundInfluentialRemote patient monitoring using artificial intelligence: Current state, applications, and challenges
Shaik, T., Tao, X., Higgins, N., Li, L., Gururajan, R., Zhou, X., & Acharya, U. R. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1485 · Cited by 580 · see citing works
“The former two uncertainties would lead to uncertainty in the modeling results can be split into data uncertainty (aleatoric uncertainty) and model uncertainty (epistemic uncertainty) (Hüllermeier & Waegeman, 2021).”
BackgroundUncertainty quantification in scientific machine learning: Methods, metrics, and comparisons
Psaros, A. F., Meng, X., Zou, Z., Guo, L., & Karniadakis, G. E. (2023) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111902 · Cited by 464 · see citing works
Cheng, S. L., Quilodrán-Casas, C., Ouala, S., Farchi, A., Liu, C., Tandeo, P., Fablet, R., Lucor, D., Iooss, B., Brajard, J., et al. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123537 · Cited by 320 · see citing works
“This procedure is also known as the variational inference (VI) [32].”
BackgroundLambert, B., Forbes, F., Doyle, S., Dehaene, H., & Dojat, M. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102830 · Cited by 318 · see citing works
A review of Earth Artificial Intelligence
Sun, Z., Sandoval, L., Crystal-Ornelas, R., Mousavi, S. M., Wang, J., Lin, C., Cristea, N., Tong, D., Carande, W. H., Ma, X., et al. (2022) · Computers & Geosciences · doi:10.1016/j.cageo.2022.105034 · Cited by 271 · see citing works
“The uncertainty associated with the inherent noise of the real data is also known as aleatory uncertainty, which is not caused by the model but irreducible (Hüllermeier and Waegeman, 2021).”
BackgroundTowards trustworthy machine fault diagnosis: A probabilistic Bayesian deep learning framework
Zhou, T., Han, T., & Droguett, E. L. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108525 · Cited by 268 · see citing works
Nemani, V. P., Biggio, L., Huan, X., Hu, Z. Y., Fink, O., Tran, A., Wang, Y., Zhang, X., & Hu, C. K. (2023) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2023.110796 · Cited by 255 · see citing works
Ding, R. F., Chen, J., Chen, Y., Liu, J., Bando, Y., & Wang, X. (2024) · Chemical Society Reviews · doi:10.1039/d4cs00844h · Cited by 252 · see citing works
Machine Learning Meets with Metal Organic Frameworks for Gas Storage and Separation
Altıntaş, Ç., Altundal, Ö. F., Keskın, S., & Yıldırım, R. (2021) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.1c00191 · Cited by 239 · see citing works
Iqbal, S., Qureshi, A. N., Li, J., & Mahmood, T. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09899-9 · Cited by 233 · see citing works
“In Bayesian uncertainty quantification strategies, the learning sample is used to approximate the posterior probability distribution [412].”
BackgroundMethodClassifier calibration: a survey on how to assess and improve predicted class probabilities
Silva Filho, T., Song, H., Perello-Nieto, M., Santos-Rodriguez, R., Kull, M., & Flach, P. (2023) · Machine Learning · doi:10.1007/s10994-023-06336-7 · Cited by 228 · see citing works
“…training set is limited, uncertainty brought about by model mismatch, or uncertainty regarding whether the model should abstain when there might be novel target values that were not seen in the training set – see (Hüllermeier & Waegeman, 2021) for a recent survey on uncertainty in machine learning.”
BackgroundHow to measure uncertainty in uncertainty sampling for active learning
Nguyen, V.-L., Shaker, M., & Hüllermeier, E. (2021) · Machine Learning · doi:10.1007/s10994-021-06003-9 · Cited by 198 · see citing works
Self-citationTian, J., Chen, C., Shen, W., Sun, F., & Xiong, R. (2023) · Energy storage materials · doi:10.1016/j.ensm.2023.102883 · Cited by 196 · see citing works
Challenges in data-driven geospatial modeling for environmental research and practice
Koldasbayeva, D., Tregubova, P., Gasanov, M. E., Zaytsev, A., Petrovskaia, A., & Burnaev, E. V. (2024) · Nature Communications · doi:10.1038/s41467-024-55240-8 · Cited by 198 · see citing works
A Survey of Machine Unlearning
Nguyên, T. T., Huynh, T. T., Ren, Z. X., Nguyen, P. L., Liew, A. W., Yin, H., & Nguyen, Q. V. H. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3749987 · Cited by 303 · see citing works
“metric that measures how much we know about the optimal hypothesis in the parameter space [72] That is, are we certain that the current model parameters are optimal for a given dataset?).”
Mortality Attributable to Ambient Air Pollution: A Review of Global Estimates
Pozzer, A., Anenberg, S. C., Dey, S., Haines, A., Lelieveld, J., & Chowdhury, S. (2022) · GeoHealth · doi:10.1029/2022gh000711 · Cited by 198 · see citing works
“10.1029/2022GH000711 18 of 25 new insights and to significantly reduce the uncertainties associated with mortality estimates (Hüllermeier & Waegeman, 2021).”
BackgroundInfluentialA prognostic driven predictive maintenance framework based on Bayesian deep learning
Zhuang, L., Xu, A., & Wang, X. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109181 · Cited by 184 · see citing works
“Without uncertainty quantification, these deep learning-based methods may find it difficult to provide information on the reliability and creditability of their prognostics [15,16].”
BackgroundInfluentialA review of predictive uncertainty estimation with machine learning
Tyralis, H., & Papacharalampous, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10698-8 · Cited by 198 · see citing works
Lin, Y., & Li, G.-H. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3156965 · Cited by 166 · see citing works
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168 · see citing works
Thelen, A., Huan, X., Paulson, N. H., Onori, S., Hu, Z., & Hu, C. K. (2024) · npj Materials Sustainability · doi:10.1038/s44296-024-00011-1 · Cited by 170 · see citing works
Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges
Zhou, Y., Liu, L., Zhao, H., López‐Benítez, M., Yu, L., & Yue, Y. (2022) · Sensors · doi:10.3390/s22114208 · Cited by 164 · see citing works
“The calibration method is mainly concerned with the aleatoric part of the overall uncertainty [192].”
BackgroundMethodA deep learning approach for complex microstructure inference
Durmaz, A. R., Müller, M. J., Lei, B., Thomas, A., Britz, D., Holm, E. A., Eberl, C., Mücklich, F., & Gumbsch, P. (2021) · Nature Communications · doi:10.1038/s41467-021-26565-5 · Cited by 158 · see citing works
Drukker, K., Chen, W., Gichoya, J. W., Gruszauskas, N. P., Kalpathy-Cramer, J., Koyejo, S., Myers, K. J., Sá, R. C., Sahiner, B., Whitney, H. M., et al. (2023) · Journal of Medical Imaging · doi:10.1117/1.jmi.10.6.061104 · Cited by 149 · see citing works
“61 Some research has been done to develop “ reliability maps ” (which are not actually spatial maps but rather a measure of “ reliability ” across the posterior probability output).”
BackgroundCritical review of machine learning applications in perovskite solar research
Yılmaz, B., & Yıldırım, R. (2020) · Nano Energy · doi:10.1016/j.nanoen.2020.105546 · Cited by 140 · see citing works
Active Learning for Deep Object Detection via Probabilistic Modeling
Choi, J., Elezi, I., Lee, H.-J., Farabet, C., & Alvarez, J. M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01010 · Cited by 135 · see citing works
“tant in active learning, as it allows the deep learning models to know about their limitations [21, 38], i.”
BackgroundHuang, L., Ruan, S., Xing, Y.-C., & Feng, M.-L. (2024) · Medical Image Anal. · doi:10.1016/j.media.2024.103223 · Cited by 90 · see citing works
Zhao, X., Fang, Y., Min, H., Wu, X., Wang, W., & Teixeira, R. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121358 · Cited by 121 · see citing works
Light-sheets and smart microscopy, an exciting future is dawning
Daetwyler, S., & Fiolka, R. P. (2023) · Communications Biology · doi:10.1038/s42003-023-04857-4 · Cited by 115 · see citing works
Current and future directions in network biology
Žitnik, M., Li, M. M., Wells, A., Glass, K., Gysi, D. M., Krishnan, A., Murali, T. M., Predrag Radivojac, A., Roy, S., Baudot, A., et al. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae099 · Cited by 118 · see citing works
Characterizing Uncertainty in Machine Learning for Chemistry
Heid, E., McGill, C. J., Vermeire, F. H., & Green, W. H. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.3c00373 · Cited by 113 · see citing works
Data-driven science and machine learning methods in laser–plasma physics
Döpp, A., Eberle, C., Howard, S., Irshad, F., Lin, J., & Streeter, M. J. V. (2023) · High Power Laser Science and Engineering · doi:10.1017/hpl.2023.47 · Cited by 114 · see citing works
A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook
Liu, M., Yurtsever, E., Fossaert, J., Zhou, X., Zimmer, W. M. X., Cui, Y., Žagar, B. L., & Knoll, A. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3394735 · Cited by 114 · see citing works
“The uncertainty can be referred to two types: a) aleatoric uncertainty, where the uncertainty like noise stems from the data, and b) epistemic uncertainty refers to uncertainty caused by unawareness about the best model [211].”
BackgroundBeyond generalization: a theory of robustness in machine learning
Freiesleben, T., & Grote, T. (2023) · Synthese · doi:10.1007/s11229-023-04334-9 · Cited by 106 · see citing works
“Relevant uncertainty targets in ML are the same as for robustness targets: predictions, performance, and explanations—and many papers in the field evaluate uncertainty for these targets (Hüllermeier & Waegeman, 2021; Molnar et al., 2021; Nadeau & Bengio, 1999).”
BackgroundBayesCap: A Bayesian Approach to Brain Tumor Classification Using Capsule Networks
Afshar, P., Mohammadi, A., & Plataniotis, K. N. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3034858 · Cited by 105 · see citing works
“It is worth mentioning that through the development of the BayesCap framework, we have tried to capture the uncertainty in the model itself, referred to as the epistemic uncertainty [20], [33].”
BackgroundYang, K., Tang, X., Li, J., Wang, H., Zhong, G., Chen, J., & Cao, D. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3270887 · Cited by 103 · see citing works
Evaluating pointwise reliability of machine learning prediction
Nicora, G., Rios, M., Abu–Hanna, A., & Bellazzi, R. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.103996 · Cited by 98 · see citing works
Risks and uncertainties in carbon capture, transport, and storage projects: A comprehensive review
Mahjour, S. K., & Faroughi, S. A. (2023) · Gas Science and Engineering · doi:10.1016/j.jgsce.2023.205117 · Cited by 97 · see citing works
A Survey on Uncertainty Quantification Methods for Deep Learning
He, W., Jiang, Z., Xiao, T.-S., Xu, Z.-L., & Li, Y.-K. (2023) · ACM Computing Surveys · doi:10.1145/3786319 · Cited by 201 · see citing works
“…the discrepancy between prediction and ground-truth, a learning problem aims to find the best hypothesis in the hypothesis space that minimizes the loss [13, 69]: In practice, the model is fitted by minimizing the empirical risk [120], defined as the average loss over the training data D : ˜ ℎ =…”
Karim, N., Mithun, N. C., Rajvanshi, A., Chiu, H.-P., Samarasekera, S., & Rahnavard, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02310 · Cited by 97 · see citing works
Active learning of deep surrogates for PDEs: application to metasurface design
Pestourie, R., Mroueh, Y., Nguyen, T. V., Das, P., & Johnson, S. G. (2020) · npj Computational Materials · doi:10.1038/s41524-020-00431-2 · Cited by 94 · see citing works
Wang, H., Shao, W., Sun, C., Yang, K., Cao, D.-P., & Li, J. (2024) · Engineering · doi:10.1016/j.eng.2023.10.011 · Cited by 102 · see citing works
Deep Learning in Population Genetics
Korfmann, K., Gaggiotti, O. E., & Fumagalli, M. (2023) · Genome Biology and Evolution · doi:10.1093/gbe/evad008 · Cited by 87 · see citing works
“…simulations explored in the literature include testing a network trained on misspecified models (e.g. Flagel et al. 2018; Torada et al. 2019; Adrion et al. 2020), and deploying it on known cases of selection and neutrality (Isildak et al. 2021) to quantify false positive and false negative rates.”
BackgroundMethodInfluentialSynthetic data in biomedicine via generative artificial intelligence
van Breugel, B., Liu, T., Oglić, D., & van der Schaar, M. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00245-7 · Cited by 97 · see citing works
“Many of these issues can be traced back to the core difficulties of synthetic data generation: quantifying the quality (including factual correct ness) and the authenticity, particularly for medical applications.”
Bayesian-EUCLID: discovering hyperelastic material laws with uncertaintiesPreprint
Joshi, A. P., Thakolkaran, P., Zheng, Y., Escande, M., Flaschel, M., Lorenzis, L. D., & Kumar, S. (2022) · arXiv (Cornell University) · doi:10.1016/j.cma.2022.115225 · Cited by 89 · see citing works
“those due to noise in the displacement field data (Hüllermeier and Waegeman, 2021) which is automatically estimated by the hierarchical Bayesian model.”
BackgroundMethodMethodology and evaluation in sports analytics: challenges, approaches, and lessons learned
Davis, J. J., Bransen, L., Devos, L., Jaspers, A., Meert, W., Robberechts, P., Haaren, J. V., & Roy, M. V. (2024) · Machine Learning · doi:10.1007/s10994-024-06585-0 · Cited by 102 · see citing works
“To increase trust, one recent trend in machine learning is for learned models to quantify the uncertainty associated with their predictions (Hüllermeier & Waegeman, 2021).”
A Deeper Look into Aleatoric and Epistemic Uncertainty Disentanglement
Valdenegro-Toro, M., & Saromo, D. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00157 · Cited by 86 · see citing works
“There are two kinds of uncertainty [8,12,14]: aleatoric or data uncertainty, and epistemic or model uncertainty.”
BackgroundMontero, A. M. B., Bibal, A., Huet‐Dastarac, M., Draguet, C., Valdés, G., Nguyen, D. T. D., Willems, S., Vandewinckele, L., Holmström, M., Löfman, F., et al. (2022) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac678a · Cited by 87 · see citing works
“In this context, uncertainty is typically categorized in two types: aleatoric and epistemic uncertainty ( Anon 2009, Hüllermeier and Waegeman 2021 ) .”
BackgroundMortier, T., Wieme, A. D., Vandamme, P. A. R., & Waegeman, W. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.11.004 · Cited by 84 · see citing works
“…spectrum S (cid:4) , one can measure the total uncertainty in a prediction by means of the Shannon entropy of the posterior predictive distribution [45]: The posterior predictive distribution can then be written as follows: with the posterior distribution over the weights given by Bayes’ rule: P ð…”
MethodSelf-citationExplainable uncertainty quantifications for deep learning-based molecular property prediction
Yang, C.-I., & Li, Y. (2023) · Journal of Cheminformatics · doi:10.1186/s13321-023-00682-3 · Cited by 86 · see citing works
“These uncertainty quantification methods are designed to model either or both aleatoric and epistemic uncertainties [31–33], which refer to the irreducible and reducible parts of the uncertainty [32], respectively.”
Ayodeji, A., Amidu, M. A., Olatubosun, S. A., Addad, Y., & Ahmed, H. (2022) · Progress in Nuclear Energy · doi:10.1016/j.pnucene.2022.104339 · Cited by 81 · see citing works
“In general, two broad types of uncertainty exist which are aleatory and epistemic uncertainties (Hüllermeier and Waegeman, 2021).”
Pelizari, P. A., Geiß, C., Aguirre, P., María, H. S., Peña, Y. M., & Taubenböck, H. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.07.004 · Cited by 80 · see citing works
A survey on epistemic (model) uncertainty in supervised learning: Recent advances and applications
Zhou, X., Liu, H., Pourpanah, F., Zeng, T., & Wang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.119 · Cited by 81 · see citing works
“Recently, Hullermeier and Waegeman [20] emphasized the significance of identifying aleatoric and epistemic uncertainty separately in machine learning.”
BackgroundResultInfluentialSalvi, M., Seoni, S., Campagner, A., Gertych, A., Acharya, U. R., Molinari, F., & Cabitza, F. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105846 · Cited by 84 · see citing works
Large-scale evaluation of k-fold cross-validation ensembles for uncertainty estimation
Dutschmann, T., Kinzel, L., ter Laak, A., & Baumann, K. (2023) · Journal of Cheminformatics · doi:10.1186/s13321-023-00709-9 · Cited by 75 · see citing works
Monte Carlo DropBlock for modeling uncertainty in object detection
Yelleni, S. H., Kumari, D., Srijith, P. K., & Mohan, C. K. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110003 · Cited by 130 · see citing works
Artificial intelligence for science in quantum, atomistic, and continuum systems
Zhang, X., Wang, L., Helwig, J., Luo, Y., Fu, C., Xie, Y., Liu, M., Lin, Y., Zhao, X., Yan, K., et al. (2025) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000115 · Cited by 123 · see citing works
Fan, X., Zhang, X., & Yu, X. B. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109088 · Cited by 71 · see citing works
Akselrud, C. A. (2024) · Fisheries Research · doi:10.1016/j.fishres.2024.107161 · Cited by 77 · see citing works
Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models
Huang, Y., Song, J., Wang, Z., Zhao, S., Chen, H., Juefei-Xu, F., & Ma, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519464 · Cited by 132 · see citing works
“…open source initiatives, NGOs, and businesses may all be negatively affected by untrustworthy LLMs. Uncertainty estimation, aimed at gauging the confidence level of model outputs [26], [27], [28], stands out as a promising approach for identifying risks in general Machine Learning (ML) models.”
Yu, X., Chai, C., Li, G., & Liu, J. (2022) · Proceedings of the VLDB Endowment · doi:10.14778/3565838.3565846 · Cited by 70 · see citing works
“Uncertainty learning Uncertainty is a very important research topic in machine learning [15, 33].”
BackgroundInfluentialTellamekala, M. K., Amiriparian, S., Schuller, B. W., André, E., Giesbrecht, T., & Valstar, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3325770 · Cited by 75 · see citing works
“In particular, we aim to estimate the aleatoric component of uncertainty [1] associated with different modalities for improved emotion recognition performance.”
BackgroundKarkaria, V. N., Tsai, Y.-K., Chen, Y.-P., & Chen, W. (2025) · Engineering Optimization · doi:10.1080/0305215x.2024.2434201 · Cited by 78 · see citing works
“The first step involves differentiating and quantifying the two main types of uncertainties: epistemic (model uncertainty) and aleatoric (inherent randomness) (Hüllermeier and Waegeman 2021).”
Uncertainty-aware deep learning in healthcare: A scoping review
Loftus, T. J., Shickel, B., Ruppert, M. M., Balch, J. A., Ozrazgat-Baslanti, T., Tighe, P. J., Efron, P. A., Hogan, W. R., Rashidi, P., Upchurch, G. R., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000085 · Cited by 73 · see citing works
“Epistemic uncertainty can be seen as a lack of information about the best model and can be reduced by adding more training data [13].”
BackgroundHoel, C.-J., Wolff, K., & Laine, L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3251376 · Cited by 69 · see citing works
“and epistemic uncertainty [1], [2], where both are important to consider in many decision-making problems.”
BackgroundLymphoma segmentation from 3D PET-CT images using a deep evidential network
Huang, L., Ruan, S., Decazes, P., & Denœux, T. (2022) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2022.06.007 · Cited by 69 · see citing works
“In spite of the excellent performance of deep learning methods, the issue of quantifying prediction uncertainty remains [17].”
BackgroundYu, H., Zhang, H., Zhang, Z., & Yang, S. (2025) · eTransportation · doi:10.1016/j.etran.2025.100420 · Cited by 73 · see citing works
Assigning confidence to molecular property prediction
Nigam, A., Pollice, R., Hurley, M. F. D., Hickman, R. J., Aldeghi, M., Yoshikawa, N., Chithrananda, S., Voelz, V. A., & Aspuru‐Guzik, A. (2021) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2021.1925247 · Cited by 68 · see citing works
“tion between the two remains an active area of research[21].”
BackgroundSer, J. D., Casillas-Pérez, D., Cornejo-Bueno, L. M., Prieto-Godino, L., Sanz, J., Casanova-Mateo, C., & Salcedo‐Sanz, S. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.108526 · Cited by 67 · see citing works
How to evaluate uncertainty estimates in machine learning for regression?
Sluijterman, L., Cator, E., & Heskes, T. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106203 · Cited by 69 · see citing works
Uncertainty Quantification of Collaborative Detection for Self-Driving
Su, S., Li, Y., He, S., Han, S., Feng, C., Ding, C., & Miao, F. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160367 · Cited by 67 · see citing works
“Uncertainty resources can be decomposed into aleatoric and epistemic uncertainty [17], [18].”
BackgroundWang, K., Shen, C., Li, X., & Lǚ, J. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3532803 · Cited by 67 · see citing works
“Hüllermeier and Waegeman [33] analyzed in detail the source of uncertainty in deep learning, and quantification methods was comprehensively summarized.”
Sheng, H., Xia, J., Wang, Y., Wang, W., Zeng, S., She, D., & Wang, G. (2022) · Water Resources Research · doi:10.1029/2022wr032183 · Cited by 66 · see citing works
“A common way to understand the model uncertainty is to consider aleatoric and epistemic uncertainty in terms of its sources (Hora, 1996; Hüllermeier & Waegeman, 2021).”
BackgroundLearning curves for decision making in supervised machine learning: a survey
Mohr, F., & van Rijn, J. N. (2024) · Machine Learning · doi:10.1007/s10994-024-06619-7 · Cited by 66 · see citing works
Monitoring machine learning models: a categorization of challenges and methods
Schröder, T., & Schulz, M. (2022) · Data Science and Management · doi:10.1016/j.dsm.2022.07.004 · Cited by 70 · see citing works
Knowledge graph enhanced retrieval-augmented generation for failure mode and effects analysis
Bahr, L., Wehner, C., Wewerka, J., Bittencourt, J., Schmid, U., & Daub, R. (2025) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2025.100807 · Cited by 84 · see citing works
“For our results, we chose an actual FMEA dataset from a production of high-voltage systems (HVS) from prismatic battery cells.”
Cruz, B. G. S., Husch, A., & Hertel, F. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1216163 · Cited by 69 · see citing works
“The literature also refers to these types of uncertainty as systematic, aleatoric and epistemic, (Hüllermeier and Waegeman, 2021; Gal et al., 2022).”
BackgroundXu, Z., Zhao, K., Wang, J., & Bashir, M. B. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102806 · Cited by 63 · see citing works
A Survey of Machine Unlearning
Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W.-C., Yin, H., & Nguyen, Q. V. H. (2022) · arXiv · doi:10.48550/arxiv.2209.02299 · Cited by 193 · see citing works
“metric that measures how much we know about the optimal hypothesis in the parameter space [72] That is, are we certain that the current model parameters are optimal for a given dataset?).”
Niedbała, G., Niazian, M., & Sabbatini, P. (2021) · Frontiers in Plant Science · doi:10.3389/fpls.2021.695110 · Cited by 62 · see citing works
Fernández, J., Chiachío, M., Chiachío, J., Muñoz, R., & Herrera, F. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104511 · Cited by 62 · see citing works
A Review of Uncertainty for Deep Reinforcement Learning
Lockwood, O., & Si, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v18i1.21959 · Cited by 63 · see citing works
Quiñones, J. J., Pineda, L. R., Ostanek, J. K., & Castillo, L. (2023) · Energy Conversion and Management · doi:10.1016/j.enconman.2023.117440 · Cited by 62 · see citing works
Finite electro-elasticity with physics-augmented neural networks
Klein, D. K., Ortigosa, R., Martínez‐Frutos, J., & Weeger, O. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.115501 · Cited by 61 · see citing works
“They have in common that they reduce the epistemic uncertainty associated with analytical approaches, in the sense that they reduce their model uncertainty [40]: For analytical models, at some point an explicit choice for some functional relationship has to”
BackgroundWang, Y., Wang, W., Ma, Z., Zhao, M., Li, W., Hou, X., Li, J., Ye, F., & Ma, W. (2023) · Water Resources Research · doi:10.1029/2023wr035194 · Cited by 61 · see citing works
“…the model itself, mainly contributed by the number of layers and hyperparameters of the neural network, and this type of uncertainty can be reduced by having more training data and more strategic hyperpa-rameter optimization (Huang et al., 2022; Hüllermeier & Waegeman, 2021; Yang et al. al., 2021).”
Sonar-Based Deep Learning in Underwater Robotics: Overview, Robustness, and Challenges
Aubard, M., Madureira, A. M., Teixeira, L. F., & Pinto, J. (2025) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2025.3531933 · Cited by 61 · see citing works
“Techniques for addressing these uncertainties have been extensively reviewed, highlighting the dual nature of predictive uncertainty within supervised learning frameworks [170], [171].”
Machine Learning Methods in CFD for Turbomachinery: A Review
Hammond, J., Pepper, N., Montomoli, F., & Michelassi, V. (2022) · International Journal of Turbomachinery Propulsion and Power · doi:10.3390/ijtpp7020016 · Cited by 60 · see citing works
“As is discussed in the reviews of Hüllermeier and Waegeman [89] and Psaros et al. [90], moving away from ‘black-box’ models and accounting for aleatoric and epistemic uncertainty will be crucial for the massive industrial application of machine learning.”
MethodUncertainty-DTW for Time Series and Sequences
Wang, L., & Koniusz, P. (2022) · European Conference on Computer Vision · doi:10.1007/978-3-031-19803-8_11 · Cited by 53 · see citing works
“Thus, we resort to the notion of the so-called aleatoric uncertainty known from a non-exhaustive list of works about uncertainty [28, 18, 15, 14, 17].”
BackgroundClinical prediction models and the multiverse of madness
Riley, R. D., Pate, A., Dhiman, P., Archer, L., Martin, G. P., & Collins, P. G. S. (2023) · BMC Medicine · doi:10.1186/s12916-023-03212-y · Cited by 63 · see citing works
“…related to the concept of epistemic uncertainty ( reducible uncertainty), which refers to uncertainty in predictions arising from the model production itself [8], rather than aleatoric uncertainty ( irreducible uncertainty) that refers to residual uncertainty that cannot be explained by the model.”
BackgroundGao, J., Wang, J., Xu, Z., Wang, C., & Song, Y. (2022) · International Journal of Fatigue · doi:10.1016/j.ijfatigue.2022.107361 · Cited by 57 · see citing works
Lin, Y., Zhang, Q., Gao, B., Tang, J., Yao, P., Li, C., Huang, S., Liu, Z., Zhou, Y., Liu, Y., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00680-y · Cited by 58 · see citing works
“…for policy optimization, yielding consistently satisfactory results in a and smart medicine applications 10,11 , it is critical to know and quantify the two prediction uncertainties, often referred to as epistemic and aleatoric uncertainties 12,13 , to make safe and correct decisions (Fig.”
BackgroundYousefirizi, F., Klyuzhin, I. S., Joo Hyun, O., Harsini, S., Tie, X., Shiri, I., Shin, M., Lee, C., Cho, S. Y., Bradshaw, T., et al. (2024) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-024-06616-x · Cited by 57 · see citing works
“Despite the acceptable performance of CNN-based approaches, there is still a challenge in quantifying prediction uncertainty [35] including the uncertainty as a the result of the difference between training and testing datasets (domain shift) and/or model uncertainty that emerges from the limited…”
Prediction of adsorption of metal cations by clay minerals using machine learning
Zhao, S., Chen, K., Xiong, B., Guo, C., & Dang, Z. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.171733 · Cited by 57 · see citing works
Busk, J., Jørgensen, P. B., Bhowmik, A., Schmidt, M. N., Winther, O., & Vegge, T. (2021) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ac3eb3 · Cited by 56 · see citing works
Dilated densely connected U-Net with uncertainty focus loss for 3D ABUS mass segmentation
Cao, X., Chen, H., Li, Y., Peng, Y., Wang, S., & Cheng, L. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106313 · Cited by 55 · see citing works
Abdar, M., Khosravi, A., Islam, S. M. S., Acharya, U. R., & Vasilakos, A. V. (2022) · IEEE Systems, Man, and Cybernetics Magazine · doi:10.1109/msmc.2022.3150144 · Cited by 55 · see citing works
VB-DeepONet: A Bayesian operator learning framework for uncertainty quantification
Garg, S., & Chakraborty, S. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105685 · Cited by 55 · see citing works
- Authors
- Hüllermeier, E., & Waegeman, W.
- Affiliations
- Paderborn University (DE)
- Ghent University (BE)
- Corresponding author
- Hüllermeier, Eyke
- ORCID
- Hüllermeier, Eyke
- Venue
- Machine Learning
- Published
- 2021-03
- Volume
- 110
- Issue
- 3
- Pages
- 457-506
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W3014596384
- Cited by
- 1,834
- References
- 111
- Citations by year
- 2026: 426 · 2025: 445 · 2024: 373 · 2023: 304 · 2022: 181 · 2021: 88 · 2020: 16
- FWCI
- 83.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Machine Learning (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, cc-by)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Projekt DEAL
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- aleatoric uncertainty, epistemic uncertainty, uncertainty quantification, probabilistic forecasting, safety-critical machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
