Bayesian Active Learning for Classification and Preference Learning

“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“We no longer need to build our entropy calculation around the type of posterior approximation (as in [MacKay, 1992, Krishnapuram et al., , Lawrence et al., 2003]) but are free to choose from many of the available algorithms.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“Information Theoretic approaches: Maximum Entropy Sampling (MES) [Sebastiani and Wynn, 2000] explicitly works in dataspace (Eqn.”

“Entropy and margin identify uncertain predictions; BALD-style criteria use stochastic disagreement [Houlsby et al., 2011, Kirsch et al., 2019]; and diversity methods seek broader feature-space coverage [Hubot-ter et al., 2025].”

“Active learning ranks candidates by uncertainty or coverage [25, 52].”

“For active learning, we compare against BALD [16], BQBC [31], QBMGP [31], and SAL [18]; all SAL variants use Hellinger distance.”

“BALD gives a Bayesian interpretation in which acquisition reflects information about model parameters (Houlsby et al., 2011; Gal et al., 2017).”

“Bayesian experimental design [17, 18], particularly Bayesian active learning by disagreement [19, 20] resolves such problems by iterating three core steps: (i) posterior parameters estimation (i.e. given a dataset, what is the most likely Wigner function?)”

“For any target pose within the manifold, we treat its rendering as the quantity of interest and measure uncertainty at that pose via the marginal predictive entropy (Shannon, 1948; Houlsby et al., 2011).”

“This additive decomposition is standard for active learning (Houlsby et al., 2011; Gal et al., 2017), selective prediction (Geifman and El-Yaniv, 2017), and OOD detection (Smith and Gal, 2018).”

“…by Disagreement (BALD): BALD selects the points that maximize the mutual information between the predicted outcome and the model parameters [47]: Intuitively, BALD seeks points where the surrogate model as a whole is uncertain in its prediction but individual posterior samples are…”

“As proposed by Houlsby et al. [2011], Eq.”

“If the predictive probabilities are C 2 b and are bounded away from zero, then Posterior averages of entropy-type quantities appear in information-theoretic active learning criteria such as BALD; see, for example, [HHGL11].”

“This can be expressed as [10]: Given p ( y | x , θ ) = N y ; Zf θ ( x ) , σ 2 2 I 2 L p , the second term in (17) is constant.”

“…Disagreement (BALD) identity isolates the epistemic component as the mutual information between the predictive distribution and the weight posterior, which vanishes when ensemble members agree and grows when they place mass on incompatible continuations [Houlsby et al., 2011, Cao and Tsang, 2021].”

“Bayesian Active Learning by Disagreement (BALD) (Houlsby 2011) and Batch Bayesian Active Learning by Disagree-ment (BatchBALD) are two active learning methods based on Bayesian neural networks.”

“Three acquisition criteria are evaluated: Balanced-Ranking, Exploitation (selecting top model predictions) [26], and Mutual Information (BALD-based, choosing compounds with lowest mutual information) [27].”

“Finally, in machine learning, active learning seeks informative training data from humans by prioritizing samples for which the model is most uncertain (Houlsby et al., 2011).”

“Active preference learning aims to minimize annotation costs by querying informative pairs, typically via information gain [12] or model disagreement [5].”

“To position REACT, we review prior work on test-time feature acquisition and highlight limitations in handling the two-phase structure of contextual onboarding followed by longitudinal monitoring.”

“Bayesian Active Learning by Disagreement (BALD) quantifies epistemic uncertainty using information gain in terms of predictive entropies (Houlsby et al., 2011).”

“Classical active learning criteria typically aim to minimize model variance globally, implicitly assuming a uniform utility over the domain (Houlsby et al., 2011).”

“Although historical methods acquire one sample at a time [12], [13], [14], since deep neural networks (DNNs) are computationally heavy, training a new model with a single training sample is highly impractical.”

“Such principles underpin active learning strategies for acquiring labeled data [21, 30, 34, 36, 50], and manifest specifically as active preference learning, which focuses on efficiently eliciting user preferences through strategic query selection.”

“As standard β values for PowerBALD, PowerPE and SoftrankBALD we used 1 as detailed in Kirsch et al. (2023) and following the evaluation in Lüth et al. (2025).”

“Third, our manual verification process (about 2.5 h per 1000 candidates) in the iterative process could be streamlined using active learning techniques, such as Bayesian active learning (Houlsby et al., 2011), to prioritize ambiguous cases, potentially cutting human effort by 50%.”

“Techniques such as Bayesian Active Learning by Dis-agreement (BALD) measure mutual information between predictions and model parameters, selecting points that maximize epistemic gain (Houlsby et al., 2011).”

“GZD-5 also used an active learning strategy where volunteers were asked to provide 40 annotations for galaxies selected by the acquisition function (BALD, [74]) and 5 otherwise [75].”

“These include uncertainty-based meth-ods ( Entropy [18], Least Confidence [21], Margin [6], Entropy Dropout [31], BALD Dropout [15]) and diversity-based methods ( Random , KMeans [29], K-Center Greedy [34]).”

“This objective is known as Bayesian Active Learning by Disagreement (BALD), since it selects a data point where model parameters under the posterior distribution disagree the most for the output label [28].”

“Bayesian approaches such as BALD [12] and BatchBALD [13] measure information gain about model parameters.”

“Before that, because the input mutation can produce multiple predictions from LLMs, which is similar to the dropout uncertainty [61], we introduce another test selection method baseline, which is based on drop-out uncertainty, Bayesian Active Learning by Disagreement (BALD) [62].”

“Moreover, softmax-BALD, introduced in (Kirsch et al., 2023) through importance-weighted sampling over the pool dataset, was also adopted in CausalBALD, more details and results are provided in App.”

“In order to evaluate the mutual information objective, we express it as the expected posterior entropy, and rearrange as [43] which can be interpreted as seeking the x for which the source prediction under the posterior leads to the most disagreement through the forward model and is known as…”

“…gained about the predictive model’s parameters ω , which is equivalent to maximizing the mutual information I between predictions and model posterior (Houlsby et al., 2011): Finally, Kirsch et al. (2019) proposed BatchBALD that extends BALD to account for interactions between datapoints within a…”

“The surrogate model is a Bayesian neural network (BNN) modeled as a Multi-Layer Perceptron with Monte Carlo dropout for uncertainty estimation [3, 20, 21].”

“8 and 9, can also maximize the mutual information [84,37,53,6] between these ensemble weights and the ground truth edge maps, i.e. , I ( Y nv ; W Rs | X nv ) and I ( Y nv ; W NRs | X nv ) , s ∈ { 1 , 2 , ..., S } .”

“When evaluated on true labels, MIC prioritises data that is simply far away from the training data, which introduces a risk of gathering data of limited relevance to the task of interest, much like the pathology of BALD (Houlsby et al, 2011) demonstrated by Bickford Smith et al (2023).”

“Given pairs [ a, b ] and [ c, d ] , the covariance is given by where k is the Mat´ern 5 / 2 kernel (the derivation for this expression is provided by Houlsby et al. [28]).”

“AL strategies can be categorized broadly into two categories [15], uncertainty-based [66], [67], [75], [76], [77], [78], [79], [80], and diversity-based strategies [56], [81].”

“BALD selects samples that maximize the difference between the model’s prediction and the posterior probability, effectively identifying samples with the richest mutual information.”

“Finally, the epistemic component H epi is obtained as the difference between total and aleatoric uncertainty, following [15], Technical details In our experiments, we use an ensemble of size T = 5 with models trained in the same way, but with different initializations on top of a corresponding…”

“It suits data with small batch size and less category contents and is easy to combine with DL. Practical example is the batch mode DAL (BMDAL) [97][98] where a batch of samples is queried.”

“Following the entropy ( H ( · ) ) decomposition popularized by Houlsby et al. (2011) and Kendall and Gal (2017), EU in the Bayesian setting is measured via mutual information : where Θ denotes the random variable of first-order probability distributions.”

“The BALD and EN methods show unsatisfactory results in the HAR and Animal datasets, respectively.”

“Closely related works [21], [23] select samples based on decision boundary proximity, [24] query based on ensemble classification, and [25] query samples based on mutual information.”

“To measure the uncertainty, we calculate the BALD score [17].”

“The work in [15] addresses the limitations with BALD [28] during test-time distribution shift to propose (J)EPIG for test-time prediction.”

“The synergy of BALD [14] with deep neural networks, amplifies acquisition performance.”

“This approach aligns with recent work in active learning and Bayesian optimization, where information-theoretic acquisition functions have been used to guide sampling strategies [21, 22].”

“In [17], BALD was analyzed for GPC and compared to other active learning algorithms including MU.”

“Here we compare the performance enhancement of Bayesian LSTM model and baseline LSTM models with four traditional acquisition functions, namely max entropy[13], mean standard deviation (Mean STD)[21], Bayesian active learning by disagreement (BALD)[19], and random acquisition.”

“…[30], which focuses on selecting samples that the model is most uncertain about, SVMs (Support Vector Machines) [31], which are used for their robustness in high-dimensional spaces, and Bayesian inference in neural networks [32], which provides a probabilistic approach to model uncertainty.”

“Another uncertainty-based policy that is the state-of-the-art among Bayesian approaches is Bayesian Active Learning by Disagreement (BALD) which selects nodes that maximise the mutual information between label predictions and model posterior [22].”

“Existing methods for deep active learning [21, 27, 32, 60] largely build upon traditional active learning methods, or a hybrid of a few methods to achieve their objective.”

“18 Furthermore, in the profiling process, a Bayesian active learning scheme 17 is used to intelligently select the top TS-A regions with the highest uncertainty.”

“[54] extend the difficulty based algorithm [7] to diversifying batch acquisitions.”

“The acquisition function BALD (Houlsby et al., 2011) was originally presented for classification tasks.”

“In [3], active learning with expected improvement criterion as acquisition function was proposed, see also [4] and the references therein; an information-theoretic approach was used in [5] to design an active learning algorithm.”

“In AL, Entropy based selection is proved to work better than many other sophisticated acquisition functions like BADGE (Ash et al. (2019)) and BALD (Houlsby et al. (2011)) when tested on a variety of tasks.”

“The hybrid works [42, 43, 44, 45, 46, 47, 48] combine both paradigms such as by measuring uncertainty as to the gradient magnitude [44] at the final layer of neural networks and selecting gradients that span a diverse set of directions.”

“As the most important part of AL, the design of the sampling algorithm generally considers three criteria: informativeness [21], [22], representativeness [23], and diversity [24].”

“The BFBrain package permits the use of smaller batches, to accommodate larger models and/or training data that a particular scalar potential might necessitate.”

“For example, BALD [16] uses a Bayesian framework to select samples based on the uncertainty of sampled networks.”

“In our context, one chooses the x ’s that maximize Equation 1 and adds those data points to the training set as in Bayesian Active Learning by Disagreement (BALD) (Houlsby et al., 2011).”

“It has been widely used over the last 20 years for data classification, mainly text and images [4, 5, 6, 7, 8, 9].”

“BALD (Houlsby et al., 2011) Furthermore, we consider 14.”

“While the pool-based strategy selects from an existing sample of generated examples, the synthesizing-based sampling re-samples from the generator based on information theoretic measures like mutual information [43, 44] or model uncertainty [45] or informativeness [13, 46] over the initial sample.”

“This is a challenge also observed in active learning [Houlsby et al., 2011].”

“BALD [55] computes model uncertainty by exploiting the variance among different dropout results.”

“Houlsby et al. (2011) selects points that maximally decreases the expected posterior entropy of the model parameters.”

“The total number of AL steps is indirectly defined by budget size, nAL = |D|/B In this study we evaluated BALD (Houlsby et al., 2011) heuristic, while applying it with and without DA applied during training time.”

“Using a batch sampling strategy, [5] queried the K samples with the best scores using consistency-based Bayesian active learning [6].”

“AL has demonstrated good performance in a variety of machine learning tasks, including computer vision tasks such as image classification, 4,5 object detection, 5–10 and image segmentation.”

“We can rearrange the mutual information as follows [15, 16, 12], I[ȳa,w|D] = H[ȳa|Xa,D]− Ep(w|D)[H[ȳa|Xa,w]], (19)”

“Houlsby et. al. in [7] showed that the expected information gain was equivalent to the mutual information between model parameters and the observed output in Eq.”

“The design of CAT is first motivated by the fact revealed in the active learning technique [15, 16] that the decision boundaries of a NN model can be strongly affected by a rare number of training examples with rich information.”

“(2) is The following Taylor expansion of order three around f = 0 is used to approximate the second term of the mutual information , according to Houlsby et al. (2011).”

“Gaussian processes (GPs) are nonparametric distributions over continuous functions that are often used in probabilistic machine learning for both supervised and unsupervised learning applications [1, 2].”

“The BALD acquisition function relies on Bayesian uncertainty (Houlsby et al., 2011) and can be utilized in view of the classification task.”

“Since this early work, direct estimation and maximization of information gain has emerged as a popular active learning method (MacKay, 1992), and has been approximated for computational tractability (Houlsby et al., 2011).”

“One can employ the trick of conditional mutual information (Houlsby et al. 2011) to calculate Eq.”

“The basic ideas include using confidence, max-entropy [7], mutual information [8], mean standard deviation [9] or variation-ratio [10] of samples as a measurement.”

“Chu and Ghahramani (2005) and Houlsby et al. (2011) study a similar problem where the entire valuation function needs to be estimated.”

“…relies on heuristic uncertainty sampling, in which data points are selected when the learner is unsure about the correct label for those points (Houlsby et al., 2011; Joshi et al., 2009; Elhamifar et al., 2013; Gal et al., 2017; Ganti & Gray, 2012; Hoi et al., 2006; Kremer et al., 2014; Yang &…”

“Our framing and the classical decomposition are the same object, not rivals [Kotelevskii et al., 2025], and our epistemic score is BALD’s [Houlsby et al., 2011; Bickford Smith et al., 2023; Nguyen et al., 2019].”

“It is also closely related to active learning, where examples are selected to be put in a batch to make learning more efficient (Houlsby et al., 2011; Gal et al., 2017), as well as contrastive learning, where positive and negative examples are balanced in a batched setting (Chen et al., 2020;…”

“Proposition 1 (Aggregation gaps) . pre-specified hypotheses Table 3 The first gap is the mutual information between inputs and predictions; an analogous entropy difference, with averaging over model uncertainty rather than inputs, is used in Bayesian active learning (Houlsby et al., 2011).”