Bayesian Active Learning for Classification and Preference Learning

“Perhaps the most ubiquitous is active learning for SVMs [Tong and Koller, 2001, Seung et al., 1992], where the volume of Version Space (VS) is used as a proxy for the posterior entropy.”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“We no longer need to build our entropy calculation around the type of posterior approximation (as in [MacKay, 1992, Krishnapuram et al., , Lawrence et al., 2003]) but are free to choose from many of the available algorithms.”

“Although the latter approach was proposed several decades ago [Lindley, 1956, Bernardo, 1979], it is not always straightforward to apply the criteria to complicated models such as nonparametric processes with infinite parameter spaces.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“[Guestrin et al., 2005, Krause et al., 2006] also use mutual information.”

“Our active learning framework for GPC can be extended to the important problem of preference learning [F¨urnkranz and H¨ullermeier, 2003, Chu and Ghahramani, 2005].”

“Information Theoretic approaches: Maximum Entropy Sampling (MES) [Sebastiani and Wynn, 2000] explicitly works in dataspace (Eqn.”

“In pool-based AL [14, 19, 41] and classification tasks, the AL problem can be defined as follows: The whole dataset at first presents a small labeled dataset part named L = { ( x j , y j ) } Mj =1 and a larger unlabeled part named U = { x i } Ni =1 , where M ≪ N , y i ∈ { 0 , 1 } is the class label…”

“On the held-out validation set, we measure the ensemble’s aleatoric uncertainty via average score entropy and epistemic uncertainty via average variance [27, 28].”

“…the Monte Carlo estimate: We then summarize total predictive uncertainty using the predictive entropy H of the posterior predictive distribution (where Y denotes the random output label) and decompose it into aleatoric and epistemic components via the entropy–mutual information identity [32–34]: .”

“We test multiple acquisition strategies: uncertainty sampling (implemented using least confidence), entropy sampling, margin sampling, bayesian sampling (implemented as least confidence), BALD (Bayesian Active Learning by Disagreement, [10]), and query-by-committee (QBC, [4]).”

“01 , perform 150 SVI steps after each expert query, and use 200 Monte Carlo samples to approximate the BALD acquisition function.”

“Finally, we study predictive learning on image and text classification benchmarks [40–42], comparing GOIMDA against uncertainty-based active learning baselines such as BALD [43–45].”

“Table 2 confirms 1 Traditional active learning methods such as BALD (Houlsby et al., 2011), CoreSet (Sener & Savarese, 2017), or BADGE (Ash et al., 2019) assume access to a static unlabeled pool and the ability to batch-select and retrain.”

“Finally, our query selection strategy is driven by a Bayesian active learning by disagreement (BALD, Houlsby et al., 2011) formulation of the expected information gain (EIG) criterion.”

“More recent research has introduced hybrid sampling strategies that combine uncertainty and diversity criteria [2, 9, 11].”

“Our formulation of information gain follows the Bayesian Active Learning by Disagreement (BALD) principle [51], defining information gain as the mutual information between predictions and model parameters.”

“However, entropy and boundary measurements are predominantly linear statistics, making it difficult to fully characterize the complex and nonlinear dependencies between generated samples and their labels [10].”

“Houlsby et al. [2] developed Bayesian Active Learning by Disagreement (BALD), whose acquisition function is exactly the mutual information between predictions and parameters.”

“BALD [5]: Quantifies epistemic uncertainty via MC dropout with M stochastic forward passes.”

“Other, more sophisticated active learning strategies exist, such as BALD [6], BatchBALD [10], ActiveMatch [29], LESS [27], TRAK [18], but we do not consider these in our experiments.”

“(2) Bayesian framework approaches focus on model parameters [4,24], often integrating Bayesian belief networks with Monte Carlo sampling [25].”

“As an ablation to demonstrate the broader benefits of our task–driven approach with other acquisition strategies, we apply our TD-FT and TD-SPLIT approaches to BALD (Houlsby et al., 2011) and Confidence Sampling (Settles, 2009), comparing to analogous unsupervised representation ( US ) setups.”

“This information gain formulation aligns with the principles of active learning and Bayesian exploration(Houlsby et al., 2011), where models seek data that maximally reduce uncertainty(Sekar et al., 2020).”

“For Bayesian active learning baselines, we evaluate EPIG [Smith et al., 2023] and BALD [Houlsby et al., 2011, Gal et al., 2017], where each of them is combined with top-B batch selection, one of the three Gumbel noise methods [Kirsch et al., 2023], or BatchBALD [Kirsch et al., 2019] for BALD.”

“This work is also in line with active learning frameworks, where learners strategically query an oracle to maximize the amount of information obtained from queries within a fixed budget or to minimize the effort to achieve a target level of information (Settles, 2009; Houlsby et al., 2011).”

“For each of the ∼ 110,000 last-70 prediction targets per architecture, we compute the model’s epistemic uncertainty using the BALD decomposition (Houlsby et al., 2011): where the expectation is approximated over M = 100 stochastic dropout-active forward passes.”

“Letting I ( r t +1 ; a ∗ | D t ) denote this mutual information, we can define acquisition rules that prioritize actions yielding high information gain about a ∗ [8], [9].”

“We also compare to a standard method popular in the literature, BALD [20].”

“Then, as a way to evaluate the uncertainty of the model, [36] introduced the use of Bayesian Convolutional Neural Networks for Active Learning (BALD) to mitigate the issue of the DL model exhibiting excessive confidence in its output results.”

“…learning of complex systems, inspired by recent advancements of Active Learning (AL) in the ML community [Wang and Shang, 2014, Ash et al., 2020, Houlsby et al., 2011, Wu et al., 2022, Settles, 2009, Ren et al., 2021], we here aim to bridge this gap by developing active learning for ROMs…”

“…points) and information-theoretic approaches, such as mutual information maximization for sensor placement (Krause et al. 2008) and Bayesian Active Learning by Disagreement (BALD) for parameter-aware selection (Houlsby, Neil and Huszár, Ferenc and Ghahramani, Zoubin and Lengyel, Máté 2011).”

“Previous approaches include uncertainty-based methods [46, 47, 48, 49, 50, 51, 52], diversity-based techniques [30, 53, 54, 55], and hybrid approaches [56, 51, 57, 58] that combine multiple criteria for sample selection.”

“In ( ) such cases, the system confirmed unpredictable observations, as explained in the literature, where the observations were considered to be virtually free from noise [35].”

“Bayesian approaches utilize Monte Carlo dropout [13, 19] or variational autoencoders (VAEs) [34] to estimate uncertainty, which is then used as a acquisition function to select the most valuable samples.”

“We compare the performance of our method (ADROIT) with recent state-of-the-art baselines, including VAAL (Sinha et al., 2019), MC-Dropout (Gal et al., 2017), Coreset (Sener & Savarese, 2017), LL4AL (Yoo & Kweon, 2019), SRAAL (Zhang et al., 2020), and TA-VAAL (Kim et al., 2021).”

“The entropy acquisition function quantifies the uncertainty of a sample using the entropy of label y as follows: (1) The BALD acquisition function evaluates the uncertainty of a sample as the mutual information between label predictions and model posterior: [H[ y | x, θ ]] .”

“These methods can be broadly categorized into two main types: uncertainty-based (Houlsby et al., 2011; Gal et al., 2017) and diversity-based approaches (Nguyen and Smeulders, 2004; Sener and Savarese, 2017; Agarwal et al., 2020).”

“This approximation is inspired by the Bayesian Active Learning by Disagreement (BALD) framework (Houlsby et al., 2011; Gal et al., 2017), where mutual information is used to quantify epistemic uncertainty.”

“However, there is no universally agreed-upon method to quantify the uncertainty associated with each unlabeled sample for a given model [22], [23].”

“…+ r n ) can be recognized as a normalization quantity for a gamma density: He et al. [2019] recognized the reciprocal gamma function 1 Γ( r n ) as where P-IG ( ξ, 0) stands for the Polya-inverse gamma distribution (details about its density and moments are included in the supplementary material.”

“A mutual information criterion can focus on the information gain in the space [11, 13] or take model improvement [14] into account, which is often considered superior to entropy.”

“Similarly, [ Houlsby et al. , 2011 ] proposed a metric originally meant for active learning but can also be utilised as an uncertainty measure for existing networks.”

“Estimating uncertainty for approximate inference via Bayesian neural networks is a common technique in active learning [ 8 , 10 ] .”

“Houlsby et al. (2011) pro-pose Bayesian Active Learning via Disagreement (BALD) and estimate uncertainty as the mutual information between the predictions and the parameters of a Bayesian model, which they reformulate as A ( x ) = H ( y | x ) − E p ( ω ) H ( y | ω, x ) .”

“BALD [11] is based on mutual information.”

“1, the sender is a learner that uses a channel to communicate with a teacher [2], [3].”

“Active learning is a paradigm in machine learning where the model can actively query the user or other sources of information to obtain new data or labels for learning purposes[22,23].”

“This difference equals the mutual information between the prediction and the posterior over the model parameters, which was first introduced for Gaussian Process Classifier (Houlsby et al. 2011) and was adopted as another measure of uncertainty in MC-dropout technique (Gal 2016).”

“A notable example of an MI-based AL method is BALD [14].”

“Let’s first introduce the definition of BALD [20], which is the basic unit of BatchBALD: where unlabelled dataset, D pool and D train are the unlabelled dataset and the labelled training dataset, respectively. x is a single cell of D pool , y is the true label of x . p ( ω , D train ) is a Bayesian…”

“A lot of studies have been proposed on active learning for image classification [5, 6].”

“As with the other preference learning dataset, we used the preference kernel of Houlsby et al. (2011).”

“…(2018), the representativeness constructed by the model Sener and Savarese (2017); Ash et al. (2019), the training effects based on current annotated results Settles et al. (2007); Houlsby et al. (2011); Toneva et al. (2018) and the hybrid of the methods above Yang et al. (2017); Zhdanov (2019).”

“## can be given as: where 𝐻(∙) is the binary entropy function 𝐻(𝑥 , which affords the approximation of the second term in closed form (Houlsby et al., 2011).”

“A tractable way of computing I(θ,y|D,x) is given in [Smith and Gal, 2018], [Houlsby et al., 2011].”

“The uncertainty sampling methods are the most popular and widely used strategies which select the difficult examples to label (Lewis and Gale, 1994; Joshi et al., 2009; Houlsby et al., 2011).”

“[ 4, 5] adopts the approach of batch acquisition and chooses BALD [6] to query top-𝐾 highest-score samples.”

“Baselines We compare CEREAL with strong active sampling baselines divided into two categories: active learning (MaxEntropy, BALD) and active testing baselines (CrossEntropy).”

“BALD (Houlsby et al., 2011) uses mutual information to measure uncertainty.”

“Alternative solutions, based on active learning [21], [22], [24]–[26], [28], [29], [31], [32], are rather preferred where users annotate very few examples of relevant and irrelevant changes according”

“In addition, there are many methods that combine the uncertainty-basedmethodwith the representationbasedmethod, e.g. the BADGE[1] method can be considered as a combination of the BALD[8]method and the Coreset[22] method, and has been experimented on several models.”

“12 can be expressed in terms of entropy of the predicted target, avoiding unnecessary posterior updates [10, 16], The model appears twice in Eq.”

“Several different types of heuristics have been proposed, such as those based on uncertainty (Kapoor et al., 2007), disagreement (Houlsby et al., 2011), diversity (Xu et al., 2007), and expected error reduction (Roy and McCal-lum, 2001).”

“Many active learning methods have been proposed [19], such uncertainty sampling [44, 28, 74, 33, 20], query-by-committee [61, 18], mutual information [24, 29, 31], core-set [60], and mean standard deviation [36, 34].”

“In this work, we used Bayesian Active Learning by Disagreement [32] (BALD) algorithm and it computes entropies in the binary output space.”

“[2017] adopt BALD [Houlsby et al., 2011] in the context of Monte Carlo Dropout to acquire datapoints that maximize the Jensen-Shannon divergence (JSD) across MC samples.”

“In the past decade, the rapid development of active learning has resulted in various sampling strategies, such as uncertainty sampling [25], query-by-committee [26] and information gain [27].”

“For the feature extraction phase, a variety of techniques were employed: Novotný & Suk [13] used Fourier descriptors, Sulc & Matas [16] developed a method using multiscale histograms, Çuğu et al. [2] produced some hand-crafted features, Wu et al. [18] extracted digital morphological features, and Kumar et al. [10]…”

“We evaluate the model uncertainty using Bayesian active learning by disagreement (BALD) [30] for the activity recognition task.”

“Where no labelled dataset of valid and invalid sequences is available, we propose a novel approach to active learning for sequential tasks inspired by classic mutual-informationbased approaches (Hernández-Lobato et al., 2014; Houlsby et al., 2011).”