Bayesian Active Learning for Classification and Preference Learning

“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“We no longer need to build our entropy calculation around the type of posterior approximation (as in [MacKay, 1992, Krishnapuram et al., , Lawrence et al., 2003]) but are free to choose from many of the available algorithms.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“Information Theoretic approaches: Maximum Entropy Sampling (MES) [Sebastiani and Wynn, 2000] explicitly works in dataspace (Eqn.”

“Houlsby et al. (2011) proposed a method of issuing queries under a Bayesian setting where the entropy of the predicted values is high and the expected entropy of predicted values for the parameter posterior distribution of the prediction model is low (i.e., low uncertainty of predicted values with…”

“We prefer it over its contemporary, Bayesian Active Learning by Disagreement (BALD) [22] , since it has less computational requirements.”

“…sample as its entropy [50]: 2) Mutual Information: , also called BALD and Jensen-Shannon Divergence, explicitly looks for large disagreement between the models (i.e., model uncertainty ) [28], [51]: where p ( e ) denotes the prediction of an individual member of the ensemble before marginalization.”

“Other information gain measures have been proposed to address shortcomings of US, such as mutual information between unlabeled samples and the model parameters [7], [14], [15], which, however, did not improve uncertainty results for CNNs, and mutual information between query candidates and unlabeled samples [16], whic…”

“05) and a BALD [29] score to quantify the prediction uncertainty.”

“BALD uncertainty results for this approach are evidenced in Figure 4.”

“Another distinction between the acquisition functions is decision-based and information theory-based [Houlsby et al., 2011].”

“Prior works have leveraged various measures to sample instances based on predictive entropy [Shannon, 2001], variation ratios [Freeman, 1965], standard deviation and more recently based on model uncertainty like Bayesian Active Learning by Disagreement (BALD) [Houlsby et al., 2011].”

“More recent approaches take a Bayesian approach and usually focus on approximating the parameter posterior [16].”

“…not predicting the most confident class, in descending order, E NTROPY (Shannon, 1948) selects samples that maximize the predictive entropy, and BALD (Houlsby et al., 2011), short for Bayesian Active Learning by Disagreement, chooses data points that maximize the mutual information be-tween…”

“In this context, Kirsch et al. (2019) [21] improve the performance of the acquisition function BALD (Bayesian Active Learning by Diverse) [17] in the batch setting (BatchBALD).”

“To measure model uncertainty, we utilise Bayesian active learning by disagreement [61], which computes the mutual”

“We then approximate the mutual information score 4 proposed by Houlsby et al. (2011) N x ∈ D q ( z | x , φ ) , i.e., a uniform mixture of N Gaussians in our case ( qz ) 5 (Mahmood & Hern´andez-Lobato, 2019).”

“Starting from a random initial design of 100 evaluations, we then use the BALD acquisition function of Houlsby et al. (2011) to sequentially improve our model over 10 data acquisition steps, each time collecting evaluations at 100 informative locations then updating the classification surrogate…”

“One popular example is stochastic regularisation techniques such as dropout during inference time, known as the Monte Carlo Dropout technique (Houlsby et al., 2011).”

“While many acquisition schemes are designed to acquire labels one at a time (Houlsby et al., 2011; Gal et al., 2017), recent work has highlighted the importance of batch acquisition (Kirsch et al.”

“Positive predictions are then filtered by the probability, mutual information and predictive entropy [Houlsby et al., 2011], screened, and stored in a curated database.”

“The vast majority of prior work on preference learning obtains at most 1 bit of information per preference query [6]–[8], [14]–[22].”

“After an initial random tone delivery, each new tone in this study was selected according to the principle of Bayesian active learning by disagreement (BALD) (Garnett et al., 2013; Houlsby et al., 2011).”

“Bayesian Active Learning by Disagreement (BALD) [18] defines an acquisition function based on epistemic uncertainty.”

“This has been previously utilised in the context of active learning in the works of [23] and [16].”

“…line of work on optimal experimental design, such as dueling bandits (Heckel et al., 2019) and Bayesian preference learning approaches (Foster et al., 2019; Gal et al., 2017; Houlsby et al., 2011), has developed methods to efficiently infer user preferences from a limited number of interactions.”

“[197] uses a greedy algorithm to find a data x that maximizes the reduction of model entropy and labels it.”

“The expected information gain is a popular acquisition function in Bayesian optimal experimental design (Lindley, 1956) and in active learning, where it is also known as BALD (Houlsby et al., 2011; Gal et al., 2017).”

“Recent works (Liu et al., 2021; Citovsky et al., 2021; Kirsch et al., 2019; Houlsby et al., 2011) combine the aforementioned heuristics: they measure uncertainty as the gradient magnitude of samples (Ash et al.”

“For active learning, BALD [27] is an established acquisition strategies that acquires points with high epistemic uncertainty [13, 34].”

“Compared with BALD, MOCU and WMOCU, the advantage of using strictly proper scoring rules in BEMPS is that they generalise expected errors to expected scores, which can be tailored for different inference tasks.”

“Epistemic uncertainty via disagreement (Houlsby et al., 2011; Gal et al., 2017; Kirsch, 2024) Epistemic uncertainty is how much the possible models disagree about the outputs.”

“…and conditional entropy for specifying explicit quantities for the total (TU), aleatoric (AU), and epistemic (EU) uncertainties associated with a predictive second-order distributions Q ∈ P ( P ( Y )) (Houlsby et al., 2011; Gal, 2016; Depeweg et al., 2018; Smith and Gal, 2018; Mobiny et al., 2021).”

“(3) (in this case the sum runs over the number of trees of the forest), to estimate the aleatoric and epistemic uncertainty, and to make RF inherently uncertainty-aware.”

“…obtain the following probability of selecting a class: The total uncertainty is commonly measured as the entropy of this probability distribution [Houlsby et al., 2011, Gal, 2016, Depeweg et al., 2018, Hüllermeier and Waegeman, 2021]: We can reformulate the total uncertainty as the expected…”

“The softmax distributions p ( y | x , ˆ ω ) obtained from these forward passes can then be used to compute either predictive entropy et al., 2011) as measures of uncertainty.”

“Uncertainty-based sampling [17, 13, 8, 10, 21] and distribution-based sampling [18, 32, 47, 33] are common methods in the pool-based scenario.”

“Ideally, information-theoretic experimental design [28, 29] would approach this problem by formulating our beliefs regarding which model is correct as a posterior probability distribution, conditioned on observed or hypothetical stimuli and responses.”

“The baseline methods include Random selection, Entropy, Least Margin, BALD (Houlsby et al. 2011; Gal, Islam, and Ghahra-mani 2017), QBC (Kuo et al. 2018), As shown in Figure 1, our method ( NoiseStability ) out-performs or is comparable to the best baselines at all active learning cycles.”

“The success of these deep learning-based methods heavily depends on large-scale annotated datasets, which are both time-consuming and costly to produce.”

“Beside AI safety, uncertainty is useful in the active learning setting in which the data collection process is expensive or time consuming [Houlsby et al., 2011, Rottmann et al., 2018].”

“According to BALD [7], the conditional entropy term H(A|ω) in Eq.”

“Grießhaber et al. [10] also examine the impact of active learning on BERT, focusing on the BALD by Houlsby et al. [11] strategy with Monte Carlo Dropout (MCDO).”

“BALD: This uncertainty-based QM uses the mutual information between the class label and the model parameters with regard to each sample for greedy selection [27], it was introduced with dropout for deep bayesian active learning[19].”

“The other one is BALD (Bayesian active learning by disagreement) (Houlsby et al., 2011), namely the mutual information between the label and model parameters, I[y,θ|x,D] = H[y|x,D]− Ep(θ”

“In BALD, pool points are chosen that maximize the mutual information between predictions and model posterior [7].”

“…, efficient data acquisition has been extensively studied for many problems, including active learning (Cohn et al., 1996; Gal et al., 2017; Houlsby et al., 2011; Ren et al., 2021; Settles, 2009; Wang et al., 2018), Bayesian optimization (Auer, 2002; Garnett, 2023; Hennig and Schuler,…”

“The mutual information between model weights and output is a well-established metric for measuring epistemic uncertainty [Houlsby et al., 2011].”

“The currently prevalent measure of predictive uncertainty is the entropy of the Bayesian model average (BMA) predictive distribution, where the epistemic component is given by the mutual information I [15, 10, 8, 17, 27].”

“It has been an open problem to estimate the prediction uncertainty of deep neural networks efficiently and reliably in multiple scenarios, which include examples of selecting samples formanuallabelinginactivelearning[63]orguidingtheQ-value functionofagentsinreinforcementlearning[64].”

“We adopt Bayesian Active Learning by Disagreement (BALD) [15] as our spatial sampling uncertainty estimation function on point cloud, as follows:”

“For instance, active learning selection functions, as well as those in curriculum learning, are largely based on choosing the next training sample to maximize the information-gain, or relatedly to reduce some notion of uncertainty over the model’s own predictions or variance in performance error [69,113].”

“Tran et al. [39] improves BALD by adding a variational autoencoder and auxiliary classifier generative adversarial network, from which a synthesized data point x ′ similar to the queried image x ∗ is generated.”

“On the other hand, entropy BALD are uncertainty-based acquisition functions that calculate an uncertainty metric using the performance of the classifier trained using the current training set.”

“[12] proposed the Bayesian active learning by disagreement (BALD) method in which the acquisition function is measured by the mutual information of the training examples with respect to the model parameters.”

“• Bregman Information (BI): which special case is BALD (Gal et al., 2017; Houlsby et al., 2011).”

“We compare different statistics over the predictions y = gWtm (x) given by the multiple forward passes, including BALD [14], Variation-Ratio, the standard deviation over the predictions std(gtm(x)) (averaged over all classes), and the mean over the predictions.”

“Popular approaches to uncertainty quantification in the literature [Houlsby et al., 2011, Gal, 2016, Depeweg et al., 2018, Smith and Gal, 2018, Mobiny et al., 2021] rely on information-theoretic measures derived from Shannon entropy [Shannon, 1948].”

“This definition of dueling information gain is an extension to preferences of the information gain, formulated for GP classifiers in [13].”

“A more sophisticated approach is BALD (Houlsby et al., 2011), which estimates the epistemic uncertainty by computing the mutual information between the model posterior distribution and the prediction: I(y, w | x,DL) = H[y | x,DL]− Ep(w|DL)(H[y | x,w]).”

“We first introduce Statistical distance-based Active Learning (SAL), which improves Bayesian active learning by generalizing previous work [24, 48] and introduces a holistic measure of disagreement between the marginal posterior predictive distribution and each conditional posterior predictive.”

“To test the effectiveness of the uncertainty estimates produced by the BayesFormer, we use the MCDropout [16] based BALD [18, 25] acquisition function (called MC-BALD) and compare it against two baselines: MC-BALD applied to the vanilla transformer model and random-sampling.”

“Houlsby et al. (2011) frame active learning as looking to maximize the information gain about the model parameters if we observe the outcome Y = y given model inputs.”

“While Expected Information Gain focuses on the formulation of the mutual information term as a reduction in model posterior uncertainty given a potential sample, in active learning, BALD was introduced as an equivalent term which focuses on the predictive disagreement (Houlsby et al., 2011).”

“The entropy of a pixel BALD (Bayesian Active learning by disagreement) [27]: chooses the image that maximizes the mutual information between the standard prediction and posterior prediction of each pixel.”

“The Total Information Gain (TIG) (MacKay, 1992) is one acquisition function that has been used to benchmark predictive marginal uncertainty (Cohn et al., 1996; Houlsby et al., 2011; Hernández-Lobato et al., 2014; Zhang et al., 2018; Gal et al., 2017; Beluch et al., 2018).”

“Standard information criteria include D-optimality (used in [Houlsby et al., 2011, 2012, Pfeiffer et al., 2012]) and E-optimality.”

“We consider the following baselines, which select the top-k points using an acquisiton function, α ( x ) : • Bayesian Active Learning by Disagreement (Houlsby et al., 2011)”

“Our quality evaluation purpose is to identify the viewpoints that can maximize the Information Gain [28]–[30] among the view-points x ∈ D candidate in comparison to D train , where D candidate represents the collection of candidate viewpoints: where H [ · ] is the entropy [29].”

“On the other hand, for model uncertainty, Bayesian Active Learning by Disagreement (BALD; Houlsby et al., 2011) acquisition function is utilized.”

“Inspired by previous active learning research [10, 12, 19], which suggests that samples with high entropy require additional attention, our active occupancy decoder refines features only for coarse voxels with the highest entropy.”

“Preference learning is a form of supervised learning that learns rankings from pairwise comparisons [24, 25, 26, 27, 28, 29].”

“, 2021), as well as popular active learning methods for improved classification (Houlsby et al., 2011) or contour line estimation (Picheny et al.”

“We measure the model uncertainty using Bayesian active learning by disagreement (BALD) [8], which quantifies mutual information between parameter posterior distribution and predictive distribution.”

“A principled approach to measuring and separating aleatoric and epistemic uncertainty, which is now widely applied in machine learning (Houlsby et al., 2011; Depeweg et al., 2018; Mobiny et al., 2017), is based on a classical information-theoretic result, according to which entropy (of a random…”

“We discuss different aspects to the problem and analyze their in fl uence on AL performance using the example of BADGE [4], BALD [13], Core-Set [34], Entropy [35], ISAL [24], LC [23], and LLOSS [42]) and the random strategy.”

“This metric is equivalent the second term of the BALD acquisition function of Houlsby et al. (2011), and is chosen as it is pre-normalised to [0, 1] unlike standard deviation or similar metrics.”

“Further, we exploit the inferred posterior over weights to guide active learning [14] methods that significantly improve performance of the system in scenarios where labeling data is expensive.”

“BatchBALD takes this issue into account and makes itself more likely to acquire a diverse cover.”

“Bayesian Active Learning is an established form of active learning that leverages the uncertainty in model parameters to select the most informative points [14, 27, 16], demonstrating relevant impact in several applications [19, 18, 17, 28, 29].”

“BALD [17] is often used as baseline in existing works.”

“We define bald (Houlsby et al., 2011) priority based on the mutual information gain: gbald(θ, x) = H[p(·|θ, x)] −∫ z PZ(dz)H[p(·|θ, x, z)], Similarly, variance uses the variation in predicted probabilities, gvariance(θ, x) =∑ c ∫ z PZ(dz) (p(c|θ, x, z)− p(c|θ, x))2 .”

“A line of research that we exclude are Bayesian approaches such as BALD (Houlsby et al., 2011), because the requirement of a model ensemble makes them computationally inefficient.”

“AL has been extensively investigated for classification tasks (Hoi et al., 2006; Joshi et al., 2009; Houlsby et al., 2011; Hahn et al., 2019; Shi and Yu, 2021), but less literature addresses AL in the regression setting (Krause et al., 2008; Garnett et al., 2014).”

“The results show that our active methods which leverage the infeasible subsequence property , sequential and incremental , outperform random-ss . random-ss only outperforms greedy , showing that a random strategy is able to find more interesting training towers than a myopic approach which simply tries to maximize the…”

“Bayesian Active Learning by Disagreement (BALD) represents the mutual information between data points and the model weights [27].”

“We compare predictive entropy and BALD [26] uncertainty measures for Bayesian Monte Carlo dropout and stochastic variational inference models.”

“GPs have many elegant theoretical properties, but their use is greatly hindered by analytic and computational intractabilities.”