Bayesian Active Learning for Classification and Preference Learning

“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“We no longer need to build our entropy calculation around the type of posterior approximation (as in [MacKay, 1992, Krishnapuram et al., , Lawrence et al., 2003]) but are free to choose from many of the available algorithms.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“Information Theoretic approaches: Maximum Entropy Sampling (MES) [Sebastiani and Wynn, 2000] explicitly works in dataspace (Eqn.”

“These components can be approximated with Bayesian models (Kotelevskii et al., 2025; Schweighofer et al., 2025), which recover the decomposition of Houlsby et al. (2011) as a special case.”

“Second, Monte Carlo dropout (Gal & Ghahramani, 2016) quantifies query-specific predictive uncertainty, from which we construct a predictable mutual-information (MI) score (Houlsby et al., 2011) scale adapts prediction-set allocation.”

“Active preference learning then chooses which comparisons to show by maximizing informativeness (e.g., expected information gain) [14, 19], which we adopt to reduce interaction burden under noisy feedback.”

“Entropy and mutual information allow splitting total uncertainty into aleatoric and epistemic (Houlsby et al. , 2011; Kendall & Gal, 2017).”

“…with the highest expected information gain, where H is the entropy and p t ( y | a ) = E q t ( θ ) p ( y | θ, a ) is the predictive under the current belief (Houlsby et al., 2011), computed exactly for the categorical families and by Monte Carlo with 64 posterior samples for the Gaussian family.”

“Information-theoretic acquisition based on the mutual information between predictions and model parameters 24 was extended to deep networks through MC dropout 25 , and deep ensembles have since become the competing standard.”

“BALD-inspired uses final-block-input hidden dropout rather than parameter dropout; BatchBALD-inspired uses a sampled joint-disagreement criterion from these stochastic predictions [1, 2].”

“It has been shown previously that sensing in a high-uncertainty region optimally increases information gain (Krause et al., 2008; Houlsby et al., 2011).”

“BO seeks a single best point [SKKS10], AL an accurate surrogate through uncertainty-or information-based queries [Mac92,SWGO00,HHGL11,BHGL19], and LSE the set of candidates above a level [BNG ∗ 05,GCHK13,BSKC16].”

“Deep active learning uses Bayesian disagreement [68–70] or representation diversity [71, 72], or combines uncertainty and diversity in gradient space [72].”

“Its task selection is related to active learning (Settles, 2009; Houlsby et al., 2011) and adaptive replay (Schaul et al., 2016), with non-degenerate group probability as the allocation objective.”

“Querying z t ∈ P t returns y t and gives BALD (Houlsby et al., 2011) selects queries using G t = SoftArgMax β t { α t ( z ) } z ∈P t , where A terminal prediction can be included as an answer action.”

“Classical acquisition functions (e.g., BALD [28]) assume an ideal oracle that always provides effective labels.”

“Methods include BALD, which uses information gain (Houlsby et al., 2011), CoreSet, which emphasizes geometric coverage (Sener and Savarese, 2018), and BADGE, which combines uncertainty and diversity (Ash et al., 2020).”

“Decision and observation Issue in the corrective loop BALD [Houlsby et al., 2011] Learner requests a label; beliefs determine its expected informativeness.”

“F1 is similar to that with an unbiased initial set, with LC, class-balanced DAL v2, BADGE and BALD the best.”

“In active learning and Bayesian optimization, one line decides where to spend evaluations (MacKay, 1992; Houlsby et al., 2011), constraining exploration for safety (Sui et al., 2015) or scalability (Eriksson et al., 2019).”

“In the language-model gating experiment (Section 6.3), the proxy’s BALD score (Houlsby et al., 2011) is thresholded to decide whether the target should abstain.”

“We inherit from pool-based active learning (Set-tles, 2009), where query-by-committee measures disagreement across hypotheses (Seung et al., 1992) and information-theoretic criteria select maximally informative queries (Houlsby et al., 2011).”

“Predictive entropy (Shannon, 1948) and mutual information (Houlsby et al., 2011) are canonical output-space baselines.”

“Information gain metrics are widely used in practice for active learning (Houlsby et al., 2011; Kirsch et al., 2019; Bickford Smith et al., 2023) and feature acquisition (Covert et al., 2023).”

“Bayesian experimental design formalizes information gathering [12]; query by committee selects through disagreement [13]; Bayesian Active Learning by Disagreement uses a probabilistic information criterion [14].”

“In the Bayesian active learning literature [Houlsby et al., 2011, Gal et al., 2017], EIG measures mutual information between the seed and potential future latent states where the inequality is from Proposition 2.”

“MI isolates disagreement among posterior draws (Houlsby et al., 2011), but it can be small even when every draw assigns a diffuse token distribution.”

“Many AL strategies have been proposed, targeting model uncertainty (Houlsby et al., 2011), batch diversity (Sener & Savarese, 2018), pool representativeness (Yehuda et al., 2022; Hacohen et al., 2022), or combinations of these.”

“053 for BALD.”

“…deviations by expected information gain (EIG) under the posterior over the candidate structures, which applies Bayesian active learning by disagreement (BALD) (Houlsby et al., 2011) takes the value 1 on the histories whose actions are α , and the posterior after h is w i ( h ) ∝ w i L M i ( h ).”

“BALD uses mutual information between predictions and model uncertainty (Houlsby et al., 2011).”

“For a finite set of model predictions, the standard information-theoretic decomposition (Houlsby et al., 2011; Kendall & Gal, 2017; Depeweg et al., 2018) defines total uncertainty (TU) as the entropy of the mean predictive distribution, AU as the mean entropy of individual predictions, and EU as…”

“…feedback was first studied in supervised preference learning (Aiolli and Sperduti, 2004; Chu and Ghahramani, 2005), and later extended to online and sequential settings motivated by applications such as human-provided feedback (Yue and Joachims, 2009; Yue et al., 2012; Houlsby et al., 2011).”

“We therefore introduce Revival , a sequence-level rejection mechanism based on the BALD score [12] computed via Monte Carlo dropout [18]: which measures predictive disagreement across stochastic parameter realizations.”

“Active acquisition selects the view whose observation is expected to most reduce uncertainty in the estimated scene parameters [10, 16].”

“24–26. where E y [ · ] denotes the expectation over categorical variable y [42].”

“…et al., 2020), Transolver (Wu et al., 2024), WDNO (Hu et al., 2025), DiffusionPDE (Huang et al., 2024b Acquisition EIG (Lindley, 1956), BALD (Houlsby et al., 2011), BAX (Neiswanger et al., 2021), PS-BAX (Cheng et al., 2024a), MF-MES (Takeno et al., 2020), DAD (Foster et al., 2021), iDAD…”

“BALD [19] maximizes mutual information between predictions and model parameters.”

“…on which hypotheses disagree (Mitchell, 1982; Seung et al., 1992; Freund et al., 1997); Bayesian active learning scores expected disagreement (Houlsby et al., 2011), ensembles use disagreement as uncertainty (Laksh-minarayanan et al., 2017), and disagreement across training runs predicts…”

“Classical work separates aleatoric and epistemic uncertainty (Kendall & Gal, 2017), and Bayesian Active Learning by Disagreement (BALD) measures parameter-induced disagreement (Houlsby et al., 2011).”

“In classification, the epistemic part is commonly the information-theoretic mutual information between predictions and model parameters (BALD) [3, 4, 8], estimated via MC-dropout [4], deep ensembles [11], or a Laplace approximation to the posterior [2, 12, 13].”

“Bayesian active learning methods such as BALD select labels informative about the model posterior [28].”

“For a known outcome pair, define the outcome-space mutual-information score where H is binary entropy [43].”

“The predictive and expected conditional entropies [26, 37] are The normalized mutual information gives the epistemic component: Larger values indicate greater disagreement among model realizations.”

“…functions are predictive entropy [19], which is a monotonic function of the posterior variance for a Gaussian, so maximizing it is equivalent to using α t ( x ) = σ 2 t ( x ) , and Bayesian Active Learning by Disagreement (BALD), which maximizes the expected decrease in predictive entropy [21].”

“It was later extended to Bayesian optimization using Gaussian process models under the methods Bayesian active learning by disagreement (BALD) (Houlsby et al., 2011), entropy search (ES) (Hennig & Schuler, 2012), and predictive entropy search (PES) (Hernández-Lobato et al., 2014).”

“Rewarding a generator by learning-progress metrics is a long standing approach, dating back to literature in artificial curiosity [29–31] and active learning [32, 33].”

“…each Monte Carlo sample we mask attention heads at a target layer and quantify disagreement among the resulting predictive distributions via the BALD mutual-information decomposition (Houlsby et al. 2011; Smith and Gal 2018), with a semantic-agreement kernel to discount surface-form disagreement.”

“…Mahaut et al., 2024 Multi-pass estimators obtain additional confidence signals through p (True) -based self-verification (Kadavath et al., 2022) or c BALD computed from stochastic forward passes (Houlsby et al., 2011). et al., 2025), it requires processing the question– response context again.”

“A critical novelty of this framework is the decomposition of uncertainty into Aleatoric and Epistemic components using Information Theory [14], [33].”

“The two terms are linked by the mutual-information identity of BALD [20], the epistemic–aleatoric decomposition [12], so ) , the expected surprise of the next state minus (1 + λ ) times its irreducible part.”

“For the first model that we augment with OPhELIA, we employ a first order approximation of Bayesian Active Learning by Disagreement (BALD) [49], which is a prominent active learning strategy which is well-suited for selecting informative training set samples for supervised learning problems [50].”

“To account for this behavior, we propose a stage-adaptive acquisition strategy that combines entropy-based and MC dropout-based uncertainty, estimated using the BALD criterion [36, 37].”

“Typical examples include entropy-based measures, least confidence sampling, and Bayesian Active Learning by Disagreement (BALD) [24].”

“However, in contin-uousdecision-makingtasks,agreedystrategyisoftenemployed, whichcanapproachoptimality[33].”

“Houlsby et al. [35] introduced Bayesian Active Learning by Disagreement (BALD), which selects samples that maximize mutual information between predictions and model parameters; in modern implementations, BALD is operationalized via MC Dropout [25], the same Bayesian-approximation principle we use…”

“We also ablate BALD (Bayesian Active Learning by Disagreement) [5] and the standard deviation of the predicted class’s maximum logit (Max-STD) across posterior samples [3] as alternative uncertainty metrics (see 5.4).”

“…through the mutual information, or equivalently the reduction in Shannon entropy, between the candidate observation and the unknown model parameters Houlsby et al. (2011), an idea originally developed for Bayesian classification and preference learning and later adapted to Gaussian process…”

“While Bayesian Active Learning by Disagreement (BALD) [Houlsby et al., 2011, Gal et al., 2017] is a dominant paradigm, applying sampling-based acquisition to large-scale regression introduces severe computational bottlenecks.”

“…predictive information gain [6] scores a candidate input x by the expected information its label carries about the labels of future test inputs, EPIG( x ) = E p ∗ ( x ∗ ) [ I ( y ; y ∗ | x, x ∗ )] , which targets predictions at test inputs rather than the latent variables υ themselves [33].”

“…Bayesian approaches which compute a softmax probabilties for N samples from the posterior, we use Bayesian Active Learning by Disagreement (BALD) [Houlsby et al., 2011]: BALD scores a candidate point x by the mutual information between its unknown label y and the current model parameters θ given…”

“…a model’s belief over possible environments — an approach termed Bayesian information gain (BIG) or Bayesian surprise (Sun et al., 2011; Little and Sommer, 2013; Itti and Baldi, 2009; Houthooft et al., 2016; Stadie et al., 2015; MacKay, 1992; Fedorov, 1972; Houlsby et al., 2011; Lindley, 1956).”

“However, unlike in the classification setting, where many authors have investigated the empirical and theoretical performance of UMs [21, 28, 56, 44], regression uncertainty has received relatively little attention.”

“They evaluated approaches such as Bayesian Active Learning by Disagreement (BALD) [14], stochastic BALD (PowerBALD) [15], and random sampling.”

“Information-theoretic approaches select pairs with highest expected information gain [4,7].”

“BALD (Houlsby et al., 2011; Kirsch et al., 2019) acquires the point maximizing I ( y ; θ | x, D ), the mutual information between observation and model parameters.”

“A common choice is to place a GP prior on the latent function f (Krause et al., 2008; Houlsby et al., 2011; Hennig & Schuler, 2012; Neiswanger et al., 2021).”

“4] in aleatoric uncertainty across methods.”

“There are several methods for assessing that uncertainty on a sample, and the most widely used in the context of deep learning are Bayesian Active Learning by Disagreement (81), least confidence, and entropy (82) scores.”

“From the second iteration onward, the training dataset is iteratively expanded by adding the top 100 molecules according to the BALD acquisition strategy.”

“To quantify epistemic uncertainty as a measure of functional diversity within our LoRA-Curve framework, we compute the Mutual Information (MI) between the curve parameter t and the predictive output y , following the Bayesian Active Learning by Disagreement (BALD) objective [9].”

“Bayesian models naturally support AL through uncertainty-based acquisition functions such as mutual information (BALD) (Houlsby et al., 2011) or predictive en-2 tropy (Shannon, 1948).”

“High posterior disagreement may indicate prompts for which the alignment data do not determine a unique safe behaviour, suggesting a role for abstention, clarification, fallback policies, or targeted red-teaming [49, 36, 84].”

“This uncertainty can be estimated from the model’s predictive probabilities [Roth and Small, 2006, Wang and Shang, 2014, Ducoffe and Precioso, 2018], or from the disagreement or variance across an ensemble of models [Houlsby et al., 2011, Gal et al., 2017, Beluch et al., 2018].”

“Information-theoretic approaches such as Bayesian Active Learning by Disagreement (BALD) use mutual information to target epistemic uncertainty directly [1, 15].”

“It builds upon the prior works on Reinforcement Learning-driven sequence de-sign [ 59 , 60 ] and Bayesian Deep Learning [ 61–63 ].”

“Similar normalizations appear in BALD [Houlsby et al., 2011], as well as in Fisher-normalized uncertainty measures dating back to MacKay [1992]; see also Kendall and Gal [2017].”

“Several lines of work produce mean-and-uncertainty outputs without calibration guarantees: Bayesian active learning by disagreement [Houlsby et al., 2011], variance decompositions for Bayesian neural networks [Depeweg et al., 2018, Kendall and Gal, 2017], and deep ensembles [Lakshminarayanan et…”

“Although these sample-based estimators expose uncertainty-related signals, they are often prohibitively expensive (Houlsby et al., 2011; Gal et al., 2017), especially in LLM-MAS, where scoring each candidate may require repeated multi-agent rollouts across multiple samples or parameter realizations.”

“…is standard in Bayesian deep learning [Gal and Ghahramani, 2016, Kendall and Gal, 2017, Lakshminarayanan et al., 2017, Depeweg et al., 2018] and central to acquisition in active learning and Bayesian optimization [MacKay, 1992, Houlsby et al., 2011, Jones et al., 1998, Srinivas et al., 2009].”

“Active learning also exploits this distinction: EU drives sample acquisition, while AU marks unin-formative or noisy data (Houlsby et al., 2011; Gal et al., 2017).”

“To reduce user effort under a fixed interaction budget, we select the next query pair ( 𝑥 ,𝑥 ) by maximizing expected information gain (IG) [15, 21].”

“…informativeness signals for U t via Monte-Carlo (MC) dropout; (2) we define a prior-aware utility that combines BALD-based mutual information (Houlsby et al., 2011) with pseudo-label class weighting to explicitly control the quality of selected samples for transfer learning under severe…”

“• BALD-based batch selection employs Bayesian active learning by disagreements(BALD) [46] to estimate uncertainty independently for each sentence, and perform batch construction externally via stratified selection.”

“In addition to predictive Entropy [66], we consider BALD [55], Power-BALD [67], Jensen–Shannon divergence (JSD) [68], and Variation Ratio [69].”

“Based on information-theoretic views of predictive uncertainty, where entropy is used as a measure of distributional uncertainty (Shannon, 1948; Houlsby et al., 2011), we quantify the model’s uncertainty via the entropy of the ensemble-averaged prediction: (2) We normalize this entropy by its…”

“Each panel shows the four active-learning acquisition strategies (entropy, BALD, Core-Set, BADGE), the random-sampling baseline, and the source’s own full-pool ceilings at 3.8K and 26K labels.”

“Information gain is a standard concept from active learning used to compare actions in terms of the expected reduction in uncertainty after observation [10, 14].”

“Higher 𝑆 info ( 𝑖 ) indicates more informative nodes for selection [13].”

“…aims to measure epistemic uncertainty using the difference be-tween the prediction’s entropy and the expected entropy over sampled model parameters (Houlsby et al., 2011; Kirsch et al., 2019; Gal et al., 2017) L EXICAL S IMILARITY (L EX S IM ) Lowest similarity between outputs sampled using…”

“…other popular active learning algorithms, e.g., active learning with Gaussian processes [26], [27], Bayesian active learning by disagreement (BALD) [28], etc., which use uncertainty (informativeness) as the primary criterion for sample selection, and may also be used for regression problems with…”