Bayesian Active Learning for Classification and Preference Learning

“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“We no longer need to build our entropy calculation around the type of posterior approximation (as in [MacKay, 1992, Krishnapuram et al., , Lawrence et al., 2003]) but are free to choose from many of the available algorithms.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“Information Theoretic approaches: Maximum Entropy Sampling (MES) [Sebastiani and Wynn, 2000] explicitly works in dataspace (Eqn.”

“For this purpose, we consider Bayesian Active Learning by Disagreement (BALD) [14,11] to score target domain samples.”

“Sampling strategies can be broadly categorized as: (i) uncertainty based, which includes selecting samples with the least confidence in its estimated most probable class [4], the smallest margin between the first and second most probable classes [21], the maximum predicted entropy [24], the minimum expected generaliza…”

“To perform active federated learning, in this study, we choose an uncertainty value as the valuation function value, Bayesian active learning by disagreement (BALD) [14].”

“This query strategy is usually implemented by the uncertainty-based sampling (Ebrahimi et al., 2019; Gal et al., 2017; Houlsby et al., 2011; Kirsch et al., 2019) and distribution-based sampling (Pinsler et al., 2019; Wei et al., 2015).”

“After estimating mean and variance of corresponding outputs, we calculate uncertainty map using Bayesian Active Learning by Disagreement (BALD) [8] metric: with the constant value , (cid:138)(cid”

“This makes sense as BALDMCD samples instances that lie in the region of uncertainty, thus reducing the size of the version space Settles [2009].”

“We take BALD [20] as an uncertainty estimate.”

“Our goal is to pick the trial that “maximizes the decrease in expected posterior entropy” of g (Houlsby et al., 2011): (x∗, h∗) = arg max (x,h) H[g|x, h,Dn]− Ey∼P (y|x,h,Dn)H[g|y, x, h,Dn], (11)”

“…this information gain is equivalent to the D-optimality criterion, which computes the log-determinant of the Fisher Information Matrix (FIM) [9,13,17]: where I t and I i respectively quantify the prior geometric certainty from W t and the new information introduced by ξ i , both…”

“A commonly employed strategy for this is Bayesian Active Learning by Disagreement (BALD), which selects samples that maximize the mutual information between model predictions and their parameters [129–131].”

“For instance, by using Monte Carlo dropout strategy to repeatedly sample model parameters, BALD [210] and BatchBALD [211] enable the quantification of the uncertainty levels of model p arameters when new samples are included.”

“…was BALD (Bayesian Active Learning by Disagree-ment), which maximizes the ‘disagreement’ in predictive posterior entropy and the hyperparameters (Houlsby et al, 2011; Kirsch et al, 2019), Here, the entropy is computed in both the parameter and output space to maximize the conditional mutual…”

“…into both the GP mean and variance, achieving comprehensive uncertainty unification, and thus provides a more accurate, interpretable, and user-aligned learning result (rightmost picture). entropy to guide query selection, leveraging information gain principles as proposed by Houlsby et al. [24].”

“Active learning iteratively selects high-uncertainty data points [18]—those with the greatest prediction errors—for training, enabling the model to optimize its performance using fewer data points [19, 20].”

“Common measures of uncertainty include entropy [49], best-versus-second best margin [50], variation ratio [37], and the mutual information between predictions and model posterior [51].”

“For sample selection, early adaption of uncertainty methods utilized sampling techniques such as Monte Carlo dropout [9] and ensembles [10], assessing the uncertainty across multiple forward passes or models using entropy or Bald [11] for each sample individually.”

“Information-based strategies that select the point that produces the largest information gain about a function’s outputs are popular and effective for global identification of f [36, 16, 12].”

“We use the tractable method suggested by [38], which roughly approximates the inferred mutual information by varying the model’s parameters to produce different predictions, computing mutual information.”

“This has lead to its adoption across various paradigms such as active learning (Houlsby et al., 2011; Gal et al., 2017), uncertainty estimation (Gal & Ghahramani, 2016; Kendall & Gal, 2017; Lakshminarayanan et al., 2017; Russell & Reale, 2021), image reconstruction (Dorta et al., 2018), human pose…”

“This enables one to estimate the mutual information (MI), I ( · , · ) between model outputs, Z , and weights, W , an established measure of epistemic uncertainty within the community [16], [22].”

“The analysis of the fingerprint selection algorithms is as follows.”

“Pool-based methods can be further divided into uncertainty-based methods (Ducoffe and Precioso 2018; Houlsby et al. 2011), representation-based methods (Sener and Savarese 2018; Caramalau, Bhattarai, and Kim 2021), and the combination of both (Kuo et al. 2018; Liu and Ferrari 2017).”

“The uncertainty sampling strategies that employ classification probabilities, Bayesian meth-ods such as variational ratios (Freeman, 1965), and deep-learning specific methods (Houlsby et al., 2011) often use epistemic (or model) uncertainty.”

“Specifically, we follow (Houlsby et al. 2011; Gal, Islam, and Ghahramani 2017) to leverage information gain of the model parameters to estimate how certain the model is to the pseudo-labeled tokens with respect to the true labels.”

“For experimental design, dense sampling of the entire multidimensional input space has been replaced with efficient active learning schemes (Houlsby et al. 2011; Settles 2009; Gardner et al. 2015a,b; Watson 2017).”

“[34] Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel.”

“…selection and only apply MC-Dropout on the level of the classifier head, similar to (Grießhaber et al., 2020), but unlike them we do not use BALD (Houlsby et al., 2011) and instead approximate p(y|s, s′) averaging the models’ predicted probabilities pφ̂(y|s, s′, w) for 50 sampled dropout masks…”

“Here, we propose to use the so-called BALD (Bayesian Active Learning by Disagreement, Houlsby et al. (2011); Gal et al. (2017)), a criterion justified from an information-theoretic perspective.”

“It was shown that the performance of this criterion, denoted as BALD , was the best.”

“(2) We can view BALD [Houlsby et al., 2011] as an approximation of BatchBALD aBatchBALD ( (x)batch, p(ω | Dtrain) ) ≤ aBALD ((x)batch, p(ω | Dtrain)) (3)”

“Information based acquisition functions play an important role in this area [12, 13].”

“Active model training was performed in this study using Bayesian active learning by disagreement or BALD [35].”

“A recent work of discrimination-based active learning [13] uses mutual information, Bayesian Active Learning by Disagreement (BALD), as discriminating criteria.”

“Information about model parameters are maximized under the posterior that disagree the most about the outcome [11].”

“…classification (Rasmussen and Williams, 2005), representation learning (Lawrence, 2005), state space modelling (Wang et al., 2005), active learning (Houlsby et al., 2011), reinforcement learning (Deisenroth, 2010), black-box optimisation (Snoek et al., 2012), and numerical methods (Mahsereci and…”

“The Bayesian active learning by disagreement (BALD) was proposed by Houlsby[17] and they applied BALD model on Gaussian Processes.”

“The goal is to achieve comparable (or better) performance with less training instances (Lewis, 1995; Houlsby et al., 2011; Siddhant and Lipton, 2018; Osband et al., 2022).”

“Alongside ATLAS’s default sampling softmax strategy, we evaluated a uniform sampling strategy, which effectively yields a discrete approximation of the Bayesian posterior necessary for information-theoretic acquisition functions like BALD [60, 27].”

“This interpretation is primarily used in active learning [ 128 ] .”

“1) Experimental Setup: Following AL approach [50], we conduct experiments on the text classification datasets News-group and Hwu64, using classical uncertainty-aware methods as baselines: MoC [51], [52], LC [53], RoC [54], Entropy [55], [56], MS [57] and BALD [28].”

“This approach underlies methods such as BALD (Houlsby et al., 2011) in Bayesian active learning.”

“Classic strategies for active learning include model prediction disagreement (Freund et al., 1997; Houlsby et al., 2011), uncertainty (MacKay, 1992; Gal et al., 2017; Kirsch et al., 2019), and dataset summarization (Sener and Savarese, 2018; Mirzasoleiman et al., 2020; Coleman et al., 2019).”

“Active learning selects maximally informative samples (often via Bayesian information-gain objectives) to reduce labeling cost (Settles, 2009; Houlsby et al., 2011).”

“For example, BALD [34], [35], [36] seeks points maximizing mutual information between predictions and model parameters, while [37] prioritizes samples expected to produce large errors.”

“Other works [31, 24] have described Bayesian active learning and adaptive design.”

“…theory proposed by Shannon [83], we utilize the information gain of model parameters to estimate the model’s certainty regarding the correctness of the pseudo-labels assigned to the code snippets, following previous studies in the field of artificial intelligence [34, 66, 93] (Section 2.1.1).”

“4) BALD [26, 32, 50]: BALD [ x ] := I ( θ, ˆ y ) .”

“…based on predicted probabilities of an ensemble such as query-by-committee (QBC) [60] (disagreement of the ensemble members), maximum entropy (MaxEnt) [22] (entropy of the averaged predictions) and BALD [33] (difference between MaxEnt and the averaged entropy of the members’ predictions).”

“…Carlo dropout is a popular approach involving randomly deac-tivating neurons of a model at inference time to estimate its epistemic uncertainty (Houlsby et al., 2011; Gal et al., 2017; Smith and Gal, 2018), while other approaches rely on a statistical analysis of the model’s features (hidden…”

“BALD [62] frequently serves as a baseline in previous studies.”

“…selection is a well-explored problem with various approaches, including active learning methods when label information is unknown [MacKay, 1992, Houlsby et al., 2011, Kirsch et al., 2019, 2023, Ash et al., 2020]; data pruning and coreset techniques for pre-training data selection [Sorscher et…”

“Modern AL approaches primarily pivot on model uncertainty (Houlsby et al., 2011; Gal et al., 2017; Shen et al., 2017; Kirsch et al., 2019; Zhao et al., 2021) or data diversity (Sener and Savarese, 2017; Shen et al., 2017; Tan et al., 2021; Hacohen et al., 2022).”

“These methods do not address the adaptation based on our dual-view preferences.”

“We illustrate the distribution of UE scores, i.e., MP, SMP, PV and BALD for all the test samples in Figure 8 and samples that are correctly predicted in Figure 9.”

“Finally, Bayesian active learning (MacKay, 1992; Houlsby et al., 2011) and Bayesian (active) causal discovery (Murphy, 2001; Tong & Koller, 2001), which can be viewed as important special cases of BED, often focus on the large batch setting, which is of particular interest for our method; notable examples of large-bat…”

“Here, the objective is to select k points that maximize the decrease in expected posterior entropy [39]: | − where H [ ] represents entropy, θ represents model parameters, and D represents the datatset.”

“correct and effective both in theory [36] and empirically [37].”

“This allows us to demonstrate the following contribution: (1) We present an extension of the Hierarchical Agglomerative Clustering (HAC) approach [2] to the Bayesian setting by replacing the min-margin with a Bayesian acquisition function, such as BALD [9].”

“A Gaussian Process (GP) is a very flexible probabilistic model andGPCs have successfully been applied to preference learning in a variety of tasks before (Chu and Ghahramani, 2005; Houlsby et al., 2011; Huszar, 2011).”

“This paper evaluates the effect of uncertainty-based acquisition functions, such as variation ratios, entropy, and BALD, on the model performance in the realm of argument strength prediction.”

“Learning incorporates the Bayesian framework and is often referred to AL with GP models (e.g. [27,54]), because GPs naturally carry the uncertainty measures.”

“Other active learning approaches propose sampling data that will cause the largest expected information gain [44], the largest expected change in the model [45]–[47], the largest expected reduction in variance [20], or the largest estimated error reduction [19], [48], [49].”

“The second criterion is a greedy objective adopted from active learning for classification [6, 10, 13], which we call Info.”

“More broadly, choosing which comparisons to collect for efficient preference recovery relates to experimental design (Graßhoff & Schwabe, 2008; Guo et al., 2018) and active learning (Jamieson & Nowak, 2011; Houlsby et al., 2011; Heckel et al., 2019; Das et al., 2025; Shen et al., 2025).”

“First, states within a trajectory are often strongly correlated, undermining their diversity or the joint information gain (Houlsby et al., 2011; Kirsch et al., 2019).”

“BALD: Following suit the Bayesian Active Learning by Disagreement (BALD) algorithm by Houlsby et al. (2011), a deep ensemble of reward models is trained for reward learning.”

“(9) is exactly the maximizer of the joint While Proposition 1 provides an intuitive connection between HIPE, EPIG, and BALD, the constituent quantities often have vastly different scales.”

“Anwar et al. (2025) develop methods for multi-task evaluation of (robot) policies with active testing, where they pool information on performance of several policies across several tasks, prioritizing tasks with high information gain leveraging Bayesian active learning (Houlsby et al. 2011).”

“Bayesian Active Learning by Disagreement (BALD) [37] and its batch version (i.e., BatchBALD) [38] selected samples that maximize the reduction in expected entropy [39] The Variational Adversarial Active Learning (VAAL) [20] proposed a pool-based active learning method, which uses VAE and…”

“We compare PromptAL with nine query strategies, including uncertainty-based strategies: Entropy [32], LC [2], and BALD [46]; diversity-based strate-14 gies: BERT-KM [11] and Core-set [47]; hybrid strategies: Patron [17], CAL [4], and BADGE [3]; and the random sampling method.”

“2) Deep Bayesian AL: This approach attempts to interpret model uncertainty rather than sample uncertainty from a Bayesian perspective [42], which combines Bayesian CNNs with AL and queries samples using Bayesian AL by disagreement [42].”

“…an excellent basis for making design decisions in such adaptive settings (Foster et al., 2021; Rainforth et al., 2024; Ryan et al., 2016), but sometimes it can be beneficial to incorporate newly acquired data into models using machine learning techniques (Houlsby et al., 2011; MacKay, 1992b).”

“To address this issue, active learning [1, 3, 6, 8, 11, 16, 18–21 , 24 ] Another approach to reduce annotation costs is weak supervision [17, 26, 27].”

“Bayesian optimization approaches have been used for psychophysics studies [28], [29], [30], [31], [32], [33], [34], [35], [36], as well as for optimization of assistive robotic devices [37], [38], [39], [40], for which the evaluation of optimization metrics is costly or the number of trials is…”

“2) BALD: Bayesian active learning by disagreement (BALD) [9] defines an acquisition function based on epistemic uncertainty.”

“…and Gale, 1994; Cohn et al., 1996; Roy and McCallum, 2001; Dasgupta et al., 2007; Dasgupta and Hsu, 2008; Settles, 2009; Beygelz-imer et al., 2010; Houlsby et al., 2011; Zhang and Chaudhuri, 2015; Shen et al., 2017; Kirsch et al., 2019; Ash et al., 2019; Brantley et al., 2020; Yuan et al., 2020;…”

“Examples of such methods include estimation of DNN loss [20, 56], entropy [12, 22], BALD [18, 24], margin sampling [37, 38], and least confidence sampling based on softmax output [41].”

“Furthermore, it allows for more efficient use of acquired knowledge for learning [5], [6], [7], and downstream computations in a hierarchical processing pipeline, among others.”

“Following Vazhentsev et al. (2022), we consider three UE techniques: sampled winning score (SWS), probability variance (PV) (Gal et al., 2017; Smith and Gal, 2018), and Bayesian active learning by disagreement (BALD) (Houlsby et al., 2011).”

“…to select the next tone intensity and frequency in order to maximize the mutual information between the expected response and the GP estimate (Houlsby et al., 2011). with the fi rst term on the right being the expected response entropy, thesecond term is the expected conditional response…”

“Bayesian Active Request adapts Bayesian active learning [60] to ODM by requesting w.p. ∼ expected reduction in entropy.”

“…and Khan (2021) to leverage uncertainty estimation from BNN to measure what the reliable unlabeled examples can be selected for training. we follow (Houlsby et al., 2011; Gal et al., 2017; Tsai et al., 2022) to leverage information gain of the model parameters to show how certain the model is to…”

“Although there are Bayesian approaches to active learning (Kapoor et al. , 2007: Houlsby et al., 2011; Houlsby et al. , 2014), they use criteria such as reduction in entropy or posterior uncertainty instead of cost information to guide the selection of which question to ask.”

“Complete objects s f ∈ S f are sampled by following a complete trajectory τ starting at s 0 and terminating at s f .”

“Research with public repositories like [16, 22] and Python packages like ModAL have started laying the groundwork but still lack ease of use, abstraction, or modern NLP”

“In the active learning literature, the BALD (Houlsby et al., 2011) score is equivalent to EIG, and can be estimated using Monte Carlo dropout neural networks (Gal et al., 2017).”