Bayesian Active Learning for Classification and Preference Learning

“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“We no longer need to build our entropy calculation around the type of posterior approximation (as in [MacKay, 1992, Krishnapuram et al., , Lawrence et al., 2003]) but are free to choose from many of the available algorithms.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“Information Theoretic approaches: Maximum Entropy Sampling (MES) [Sebastiani and Wynn, 2000] explicitly works in dataspace (Eqn.”

“From an information-theoretic point of view, active learning means finding the unlabeled points in the pool set with the highest expected information gain, which is also referred to as BALD score (Houlsby et al., 2011).”

“…computed at stage 0, the inverse ( Q t ) − 1 at each subsequent stage t can be calculated from Discussion: Our approach draws inspiration from Bayesian AL methods that aim to minimize the entropy of the learner’s parameters and their approximations(e.g. (Houlsby et al. 2011; Kirsch et al. 2019)).”

“For instance, Bayesian active learning by disagreement (BALD) selects the most critical design point according to the mutual information between the responses and model parameters.”

“To evaluate the proposed acquisition function LEA, we use a simple active learning setup [50] where an ensemble of neural networks is first trained on a small class-balanced subset of 20 inputs from the CIFAR10 training set.”

“Dataset Bayesian Coreset Ours-Greedy Ours-IHT BALD Batch BALD Random Fashion MNIST 89.53± 0.25 89.83± 0.26 89.97± 0.23 89.72± 0.23 88.85± 0.22 88.39± 0.32 CIFAR10 77.73± 0.70 78.28± 0.53 77.55± 0.83 77.61± 0.59 76.12± 0.35 76.36± 0.59 CIFAR100 42.40± 0.29 42.53± 0.34 42.30± 0.29 41.99± 0.60 42.63± 0.40 42.10± 0.24 Fig…”

“Active Learning helps reduce the amount of labeled relative comparisons by querying the oracle with informative samples (Settles, 2009; Houlsby et al., 2011).”

“(Houlsby et al. , 2011), which assesses information gain based on posterior uncertainty.”

“Evaluating existing heuristic function, BALD (Houlsby et al., 2011) for the semantic segmenta- ar X iv :2 20 2.”

“UST adopts Bayesian active learning by disagreement [Houlsby et al., 2011] using stochastic dropouts to selectively sample unlabeled examples that maximize the information gain between predictions and the model posterior.”

“To do this, we used the Bayesian Active Learning by Disagreement (BALD) algorithm, developed by Houlsby et al. (2011).”

“Standard uncertain strategies consist in choosing samples that maximize the prediction entropy (Houlsby et al., 2011; Cao & Tsang, 2021), the distance from the hyperplane in SVM (Schohn & Cohn, 2000), or the variation ratio in Query-by-committee with ensemble methods (Burbidge et al.”

“In [7], the authors showed that GP preference learning is equivalent to GP classification with a transformed kernel function.”

“A variation toGAAL, and based on previous work by [93], [94] developed a Bayesian generative active deep learning approach, performing a joint training of the generator (a variational autoencoder) and the learner, which requires smaller sample sizes and a single training stage.”

“Some studies concluded that BALD does not perform well with an imbalanced test set and in the first few samples [18, 24].”

“They extend BALD to the batch setting for neural networks with BatchBALD (Kirsch et al., 2019).”

“In the first part of this work an information theoretic active learning approach is proposed which unlike those in [9], [10] and [11] optimizes the expected log-loss regret of a test sample.”

“These include Var Ratio, Entropy, and BALD (Houlsby et al., 2011), definitions of which can be found in Appendix A.”

“Under the information-gain ob-jective [Houlsby et al., 2011], the optimal acquisition score is the mutual information | ∥ | D ] .”

“Our acquisition rule is a top-k -aware, bias-aware instance of Bayesian experimental design (Houlsby et al. 2011): it uses the posterior over membership in the top-k set, and it is computed under a judge model that contains nuisance bias parameters.”

“This joint formulation is conceptually inspired by the aleatoric–epistemic uncertainty decomposition (Houlsby et al., 2011; Depeweg et al., 2018; Kendall and Gal, 2017), establishing distributional divergence and predictive confidence as complementary axes.”

“…annotator disagreement rate; (C4) acquisition quality in active learning—a natural context for Bayesian Active Learning by Disagreement (BALD; Houlsby et al., 2011); and (C5) selective prediction via the Area Under the Risk-Coverage Curve (AURC) and mis-1 classification detection via the…”

“…2025) improves learning efficiency by arranging training in an easy-to-hard sequence; subsequent work further introduced uncertainty-based active sample selection (Houlsby et al., 2011; Gal et al., 2017; Kirsch et al., 2019), prioritizing the learning of samples with the highest information gain.”

“While BOED has long been a staple in active learning for data selection (MacKay, 1992; Houlsby et al., 2011), its application to the inference process of foundation models is nascent.”

“…influence-function estimator versus our information loss criterion. sition rule in Bayesian experimental design and active learning (MacKay, 1992; Houlsby et al., 2011), information-theoretic analyses of generalization via mutual information (Xu and Raginsky, 2017; Steinke and Zakynthinou,…”

“Houlsby et al. [2011] introduce Bayesian Active Learning by Disagreement (BALD), originally for Gaussian processes and binary classifiers, selecting data points that maximise the mutual information I ( y ; ω | x ) between labels and model parameters; this objective was subsequently adopted as the…”

“Using the variance-gated distribution q , the total (predictive) uncertainty is measured by Following standard ensemble decomposition (Houlsby et al., 2011), we define the gated aleatoric and epistemic components as To compare our variance-gated decomposition, we computed the corresponding standard…”

“To better capture epistemic uncertainty, Bayesian approximations such as Monte Carlo dropout, deep ensembles, and Bayesian Active Learning by Disagreement (BALD) have been proposed [29–31].”

“The pool-based AL paradigm commonly employs strategies grounded in uncertainty [14, 18, 26], diversity [19, 47], or a combination of both through hybrid objectives [2, 11, 33, 55].”

“This epistemic uncertainty measure, given by the mutual information (5) between parameters and data, plays a central role in Bayesian active learning , where it is used to select the most informative data points for labeling [27].”

“On this basis, EM is widely used in active learning (reducing the number of possible hypotheses as rapidly as possible) [6; 15], domain adaptation (aligning the target data distribution with the source data distribution) [7; 27; 28], and online learning (connecting entropy to error and shift) [5;…”

“While the most commonly used measures are based on the Shannon entropy (Houlsby et al., 2011), those have been criticized for having undesirable properties (Wimmer et al., 2023 Uncertainty quantification in regression.”

“AL strategies query either single data points [108] or batch data points [109].”

“The core assumption is that, for a Bayesian model or predictor ensemble, OOD inputs exhibit significant prediction disagreement, reflecting high EU (Kotelevskii et al., 2025b; Schweighofer et al., 2023; Houlsby et al., 2011; Hofman et al., 2024; Schweighofer et al., 2025).”

“To decide which points to annotate, we use the Bayesian Active Learning by Disagreement (BALD) criterion [25].”

“It is the standard formulation used by Houlsby et al. [11], Gal and Ghahramani [1], Lakshminarayanan et al. [2] and Schweighofer et al. [4].”

“…with dataset D , the amount of information we would gain about the model parameters is given by: (3) Equation (3) called Bayesian Active Learning by Disagree-ment (BALD) [5] and shows that if the model is already familiar with input x , it will gain little information by obtaining the label.”

“…particularly crucial in high-stakes domains such as healthcare and criminal justice, where the consequences of errors are severe and where legal and ethical considerations demand human oversight (Ali, Lahoti, and Gummadi 2021; Kostick-Quenet and Gerke 2022; Lykouris and Weng 2024; Guo et al. 2024).”

“…are cluster based methods for active learning to find representative and typical examples (Sener & Savarese, 2017; Hacohen et al., 2022) and uncertainty based methods that find data for which the model is either uncertain or degraded (Rahmati et al., 2024; Houlsby et al., 2011; Ash et al., 2019).”

“…intrinsic rewards, rooted in Bayesian Optimal Experimental Design (BOED) (Lindley, 1956; Pukelsheim, 2006; Foster et al., 2019; Rainforth et al., 2024) and Bayesian Active Learning (BAL) (Houlsby et al., 2011; Hanneke et al., 2014; Gal et al., 2017) principles, have been previously proposed.”

“Additional related works may be found in Appendix G .”

“• BALD Sampling : Bayesian Active Learning by Dis-agreement (BALD) [37] estimates the mutual information between model predictions and its parameters to measure uncertainty.”

“We frame this as a Bayesian active learning (BAL; Houlsby et al., 2011; Gal et al., 2017) problem—incrementally observing the unseen point that is expected to maximize the average likelihood on all the other unseen points, based on the uncertainty information provided by the given model.”

“Bayesian Active Learning by Disagreement (BALD) (Houlsby et al., 2011) treats DL model parameters as probabilistic.”

“For OOD generalization tasks, we employ the same datasets as in (Deng et al. 2023): SA (Kaushik, Hovy, and Lipton 2020), NLI (Kaushik, Hovy, and Lipton 2020), and ANLI (Houlsby et al. 2011).”

“Here, we apply a spatial sampling approach based on Bayesian Active Learning by Disagreement (BALD) [24].”

“Uncertainty is classified into aleatoric (data/irreducible) and epistemic (model/reducible) types, which can be separately estimated either via uncertainty estimation heads Nix and Weigend [1994], Kendall and Gal [2017] or Bayesian Active Learning by Disagreement Houlsby et al. [2011].”

“The difference of GPs, f ( x 1 ) − f ( x 2 ) , is itself a GP, hence it is common to directly learn the difference as a GP with a special preference kernel (Houlsby et al., 2011).”

“Let P denote the data in D S 3 and the method for seeking data point P is by [39]: where y is the prediction of data input P , and D l is the labeled data for model training.”

“• Bayesian Active Learning by Disagreement ( BALD ; Houlsby et al. 2011): where p c and p ct are the mean and individual class probabilities, respectively.”

“This could be done by selecting examples that the model is most uncertain about [20, 36, 50], examples that are most representative of the unlabeled data [2, 35], or a combination of both [49].”

“Before EDL, BALD [47] is proposed to estimate epistemic uncertainty through Bayesian network.”

“IV demonstrate that step amplification introduces only a small runtime increase to DP-AL of a few minutes across all experimental settings.”

“In the context of BAL in GPs, Houlsby et al. [20] applied Bayesian Active Learning on GP classification problem where they defined an acquisition function which estimates the quantity of mutual information between the model predictions and the model parameters.”

“Aggre-gated scores are then obtained using the BALD measure (Houlsby et al., 2011), based on which instances are sampled in a class-balanced manner.”

“While this is not guaranteed to provide an optimal solution, it is often employed for similar approaches, e.g. [ Houlsby et al. , 2011 ] , and is generally considered effective.”

“BO incorporates a prior probability distribution over the objective function that can be updated as new data are collected [39].”

“We introduce another fault detection method baseline which is based on dropout uncertainty, Bayesian Active Learning by Disagreement (BALD) [16].”

“To measure epistemic uncertainty only, the mutual information between the output y and the parameters R can be employed [11, 13, 19, 20], which, in our case, can be formalized as: represents a measure of aleatoric uncertainty.”

“A common method to estimate the information is with the Bayesian Active Learning by Disagreement (BALD) acquisition function [Houlsby et al., 2011], which estimates the mutual information between the model parameters and the labels to identify which labels to acquire.”

“MIAOD [5] proposed a reweighting method to distribute the weight of unlabeled instances, so as to achieve a balanced relationship between instance-level features and image-level features, eliminate the disadvantage of local information masking global information caused by the previous method, and…”

“BALD [23] chose data points that are expected to maximize the mutual information between predictions and model posterior.”

“Most previous AL methods rely on model predictive uncertainty or disagreement for unlabeled data, and the most uncertain data are believed to be the most informative and worthful ones to be annotated (Lewis, 1995; Houlsby et al., 2011; Margatina et al., 2021; Zhang et al., 2022a).”

“dataset (see Figure 4), we show a minority class recall improvement of 32% and an overall accuracy gain of 14% compared to the best-performing traditional AL method (BALD [36]).”

“23,27,31,32,36 We can write it as the Entropy of the average softmax vector minus the average of the Entropy of the individual softmax vectors: For object detection tasks in images or point clouds, object classification probability is not the only consideration in selecting samples to label in…”

“We use the commonly used uncertainty measure BALD in the active learning framework (Gal et al., 2017; Houlsby et al., 2011), which is de-fined the information gained of the model parameters; see Appendix G.1 for a definition.”

“We use the BALD formulation of EIG (Houlsby et al., 2011) which chooses queries based on maximizing the mutual information between the outcome and the latent parameter θ: I(y; θ|x, πn) = Hπn(x)− Eθ∼πn [ Hθ(x)] ] ,”

“In this section we use the mathematical model of [13] which is repeated here for clarity.”

“Here, we introduce kind of AL algorithm based on uncertainty theory to assist the training process of RL, called Bayesian Active Learning by Disagreement (BALD [14] AL algorithm.”

“…integrated with deep learning methods, it is used in such applications as dataset shift (Ovadia et al., 2019), adversarial data detection (Ye & Zhu, 2018), active learning (Houlsby et al., 2011), Bayesian optimization (Hernández-Lobato et al., 2014), and reinforcement learning (Janz et al., 2019).”

“This paper considers QBC based on the BALD (Bayesian Active Learning by Disagreement) method [23].”

“Mutual information (MI) between the posterior over the weights and the prediction y∗ quantifies the uncertainty in the BNN’s output [50].”

“• BALD (Bayesian Active Learning by Disagreement) (Houlsby et al., 2011; Gal et al., 2017): Choose a set of samples from an unlabeled pool that yields higher model uncertainty.”

“, 2017), BALD/Batchbald (Houlsby et al., 2011; Kirsch et al., 2019), and core-set AL via robust k-Center (Sener & Savarese, 2018).”

“Prior research conducted on the data set [2] uses active learning [3] to select the most informative galaxies for classification by volunteers, showing improvements in the accuracy of the classification task using this technique.”

“Similarly, we test the active learning with BALD [20] activation function.”

“Many active learning methods for neural networks rely on measures of the “informativeness” of a query, in the form of classifier uncertainty, margin [19,11] or information gain [18,13,21].”

“Several approaches to active learning have been proposed, including sampling the most uncertain data points [40, 63, 57, 34, 71, 25], sampling where an ensemble of models disagrees [59, 14, 42, 26, 31, 47], sampling data that will cause the largest expected information gain [29], sampling data that will cause the larg…”

“The querying methods compared are random, entropy [38], max-softmax [21], margin [35], ODIN [29], BALD [22], and vote entropy [8].”

“For the informativeness criterion we mainly consider the uncertainty sampling, including Max Entropy [15], BALD (Bayesian Active Learning by Disagreement) [7] and Least Confident [11].”

“As the listener is presented randomly sampled (or actively sampled (Houlsby et al., 2011; Naghshvar et al., 2012)) sections of audio data, a section much shorter than 5 seconds would make the task of tuning into each new example very difficult due to the absence of a reference signal.”

“Most similar to our methodology, Brochu et al. (2010) and Houlsby et al. (2011) model a user’s utility function using a Gaussian process, updating the corresponding prior after each user response, and adaptively choose questions by minimizing an estimate of posterior entropy.”

“Epistemic uncertainty is quantified information-theoretically using BALD (Houlsby et al., 2011), which measures the mutual information between the model output y and the model parameters θ conditioned on the dataset D : Intuitively, BALD isolates the information that learning the true model parameters θ would provide…”

“BALD measures the mutual information between predictions and model parameters at a given context [11].”

“The quantity estimates the mutual information between the parameters and the noisy predicate reading, where H is the binary entropy (Houlsby et al., 2011); because ϕ j is a fixed function of the observation, this information lower-bounds the information in the full observation.”

“EE is exactly the Bayesian Active Learning by Disagreement (BALD) acquisition score [20]; here it is used purely as an epistemic-uncertainty measure and no active learning is performed.”