Active Learning for Deep Detection Neural Networks

“To this end, we follow [3] and resize the logits spatially using the bilinear upsampling.”

“er must have the same size as the image. To this end, we follow [3] and resize the logits spatially using the bilinear upsampling. Each Fire Residual module follows the same architecture indicated in [12]. Also, each downsampling layer is composed of convolution and pooling layers which are applied in parallel and con…”

“[14] and Gal et al .”

“CaltechPed[11] • 640x480 resolution • 51363 Images • 20062 Boxes • Videos”

“In other words, there is a domain shift [19, 25, 28, 27] between Xl and Xu that makes the active learning procedure more challenging.”

“In other words, there is a domain shift [19, 25, 28, 27] between Xl and Xu that makes the active learning procedure more challenging.”

“[2] computed the marginal score [23] of candidate bounding boxes and integrate them using different merging functions.”

“Also, each downsampling layer is composed of convolution and pooling layers which are applied in parallel and concatenated at the output [17].”

“In other words, there is a domain shift [19, 25, 28, 27] between Xl and Xu that makes the active learning procedure more challenging.”

“This score has been previously used by [11] and [8] in the task of image classification.”

“[7, 6, 15] add a prior on the weights of image classification neural networks, sampling the weights from the dropout distribution at each evaluation.”

“[7, 6, 15] add a prior on the weights of image classification neural networks, sampling the weights from the dropout distribution at each evaluation.”

“In other words, there is a domain shift [19, 25, 28, 27] between X l and X u that makes the active learning procedure more challenging.”

“Active Learning for Object Detection (Kao et al.[5])”

“[5] formulated the selection as a convex optimization problem taking into account the similarity of selected samples in the feature space as well as their informativeness score.”

“[10] for action localization.”

“tion so as to obtain high-quality query samples. Fig.3 presents a comparison between traditional AL, RAL and DRAL pipelines. On the other hand, Active-iNAS [50] notices that most previous DAL methods [3, 5, 89] assume that a suitable DL model has been designed for the current task, meaning that their primary focus is…”

“A system with an active learning strategy may query users to provide labels for unlabeled instances [46].”

“Various approaches have been proposed to reduce these costs, such as semi-supervision [41], weak supervision [52], active-learning [3] and self-supervision [26] with task fine-tuning.”

“Researchers have attempted to overcome this barrier using active learning [1, 47], semi-supervised learning [8, 36], and weakly-supervised learning [26, 43] with limited results.”

“Active learning methods play an important role in natural language processing, computer vision, engineering systems, and other fields [20], [21], [51], [52].”

“Approaches such as active learning [1], semi-supervised learning [34], and weakly-supervised learning [39] have attempted to overcome this barrier by providing more efficient learning, but with only limited success.”

“[15] propose an image-level scoring process to rank unlabeled images for”

“Note this approach is re-lated to active learning [75, 9, 48, 2, 35], which has been used for segmentation datasets [46, 83, 89, 77, 40].”

“AL has been applied with neural networks in several fields, predominantly in classification tasks such as image recognition [16], text recognition [67], or object detection [2] (see [46] for a survey of similar AL applications), with less attention in the literature on regression of physical processes, let alone rare…”

“In recent years, pool-based active learning has been successfully applied in many deep vision tasks such as image recognition [35, 44], object detection [2], and semantic segmentation [1].”

“More recently, Aghdam et al. [22] propose an image-level scoring process to rank unlabeled images for their automatic selection.”

“Query-computing methods are further classified into applying Bayesian inference [37,28,45,24,48,76,32], ensemble learning [6,68,33,63,18], diversity learning [72,85,21,77,22,53,11] and representation learning [55,41,1,42,12,19,29,30,84,87,75,62,23].”

“Other uncertainty-based methods attempt to aggregate pixel-level scores for each image (Aghdam et al., 2019), reformulate detectors by adding Bayesian inference to estimate the uncertainty (Harakeh et al.”

“An image-level score was proposed by aggregating pixel-wise scores for active selection [19].”

“…has been applied in various fields, such as inverse materials and devices design (Chen & Gu, 2020; Pestourie et al., 2020; Tang, Kojima, et al., 2020), speech emotion recognition (Abdelwahab & Busso, 2019; Zhao & Ma, 2013), object detection (Aghdam et al., 2019; Choi et al., 2021), and so on.”

“In [28] 9 landmarks are labeled for each image (two for the eyes, one for the nose and three for the each ear), which is sufficient for detecting and analyzing general information about animal faces (tilt of the head or direction of movement), but is not enough for analyzing complex movements of…”

“Effectiveness of CLAUS scoring: We also evaluate different scoring functions (random, equidistant, entropy [1], uncertainty [14]) paired with the proposed STeW loss in Figure 7.”

“[81] compute pixel-level uncertainty scores by the divergence among predictions at different levels of decoder for each pixel and their neighborhoods.”

“Active learning has been successfully applied to several computer vision applications, such as: image classification [2], [3], object detection [4], [5], Visual Question Answering (VQA) [6], remote sensing [7] and action localization [8].”

“In recent years, some researchers shifted their attention to the downstream visual recognition tasks such as object detection [2,11,20,26,36,44,60] and semantic segmentation [47,57,59].”

“Active learning is used to ask experts for providing annotation for unlabeled samples [43], [44].”

“The application of entropy is common among active deep learning applications [26], where the training of an ensemble of classifiers is often too expensive.”

“AL is extended for 2D object detection [2,11,17,51], and 3D object detection from LiDAR [9,13,34].”

“For instance, one approach, known as active learning [1, 16], seeks to minimize manual annotation effort on a large dataset while maximizing the model’s performance [8, 33, 39]; In specific, prior HITL methods for pose estimation [8, 10, 24] have involved actively selecting and labeling informative…”

“While accelerating improvement is happening in single-stage object detectors, much of the current work on active learning for object detection focuses on two-stage detectors [1,8,12,17,19,23,28], or requires training auxiliary models [27].”

“…been extensively used in computer vision, and have been shown to reduce labelling effort and increase learning speed in object recognition tasks (Aghdam et al., 2019; Kapoor et al., 2007; Vijayanarasimhan and Grauman, 2011), classification problems (Li and Guo, 2013; Wang et al., 2016), and…”

“[2] in 2D object detection.”

“Another solution is simply aggregating the uncertainty of instances or pixels as the image-level uncertainty [19].”

“Most studies on deep active learning in computer vision are based on mono-modal RGB images, including the most recent ones in deep active learning for object detection [15, 16] and semantic segmentation [17].”

“[26] proposed an active learning algorithm which is suitable to detector based on convolutional neural network.”

“Aghdam et al. [32] proposed an image-level scoring process as indicator for the active learning of convolutional neural networks.”

“Aghdam et al. [46] believe that the predictions around an uncertain region, i.e., false positive and false negative, should be unstable.”

“AL methods for object detection measure uncertainty of a detected box through either classification or localization uncertainty [2, 4, 8, 16, 26, 47].”

“There are also AL researches focusing on other computer vision tasks, like object detection (Yuan et al. 2021; Aghdam et al. 2019), person re-Identification (Liu et al.”

“b) Active Learning for Object Detection: In the context of active learning for object detection, specific metrics related to characteristics of the underlying network can be applied [28].”

“We evaluate random, equidistant, entropy-based [53] and uncertainty-based [73] selection methods as baselines and compare with our approach in Table 1.”

“Although many studies exist for enhancing the quality of a dataset, fully automated DQC methods either suffer from low accuracy (e.g., unsupervised ML methods) or rely on a large number of quality labels (e.g., supervised ML methods).”

“Compared to most of other AL frameworks (Yang et al., 2017; Yoo and Kweon, 2019; Liu et al., 2020a; Shen et al., 2020; Aghdam et al., 2019) which need to iterate the full unannotated dataset for each time of the suggestion to sample most uncertain and/or most representative ones, the proposed method directly suggests…”

“Active learning has widespread applications in various fields such as text processing [27][35], image categorization [6][10][11], and object detection [1][8][23].”

“Deep active learning (DeepAL) combines AL strategies with deep networks to iteratively identify the most informative unlabeled samples for annotation [10], [11].”

“This challenge can necessitate active learning (Aghdam et al., 2019; Jian et al., 2021), in which a model identifies selected unlabelled datapoints and alerts a human expert to label them.”

“These strategies have been successfully applied to a series of traditional computer vision tasks, such as image classification [11,9], object detection [1,2], image retrieval [31], remote sensing [6], and regression [13].”

“Active learning has been widely used in image classification, but it is rarely used in the field of object detection [22–24].”

“…than positional— information in outputs (i.e. Brust et al. (2018) considers the marginal score of the object category predictions and does not use the bounding boxes, and Aghdam et al. (2019) indirectly uses the positional information based on a pixel map for image-level aggregation).”

“For instance, active learning techniques [27–30] focus on automatically finding the a priori best training images for their posterior manual labeling out of a large number of unlabelled ones; self-labeling techniques [31–33] focus on progressively self-labeling images by refining increasingly…”

“From other studies, methods tailored for object detection tend to outperform our baselines [1, 7, 14, 20, 39, 53], so we recommend considering them in addition to mentioned methods.”

“By selecting and labeling high-quality samples, this technique not only enhances model performance but also reduces data costs in various fields [19], [20], [21].”

“Dierent active learning algorithms including baseline random sampling, entropy Roy et al. (2018), ALDD Aghdam et al. (2019), and CALD Yu et al. (2021) are compared.”

“5,39 Kao et al. 22 and Roy et al. 38 designate one set of boxes to be the reference boxes and guarantee a corresponding box from the other inference results.”

“Due to its huge potentiality, active learning has been widely applied to various tasks, such as image classification [1], recommendation systems [2], object detection [3], semantic segmentation [4], and so on.”

“Currently, pool-based active learning has achieved promising results on multiple vision tasks, such as image recognition [35], [36], object detection [37], and semantic segmentation [38].”

“Recent works can also be found on active learning for DNN based object detectors (Choi et al., 2021; Aghdam et al., 2019), where the focus is on adaptations of active learning to existing object detection frameworks.”

“Classification uncertainty-focused meth-ods utilize the predicted class probability distribution as an estimate of the model’s confidence for each sample [2,4,16,27,39].”

“Confidence estimation is a common manner to calculate the uncertainty, and the traditional uncertainty sampling methods, including least confidence (LC) [8], [9], margin sampling (MS) [20], ES [1], [2], etc., also belong to this.”

“More broadly, active learning has been adopted in various prior works to accelerate data labeling in image classification (Gal, Islam, and Ghahramani 2017) and object detection (Aghdam et al. 2019) tasks.”

“Diversity is an effective way to avoid sampling homogenization, but it is neglected in recent state-of-the-art active learning methods [1, 4, 9, 26].”

“However, these advances rely heavily on large-scale annotated datasets a scarce resource in clinical domains where expert labeling is time consuming, expensive, and requires specialized medical knowledge.”

“Recently, deep active learning has been attracting increasingly more attention in many areas, including classification [37], [38], [39], [40], object detection [41], [42], [43], [44], and semantic segmentation [45], [46], [47], [48].”

“The uncertainty-based approaches [10]−[16], on the other hand, focus on selecting samples with the highest uncertainty in the model prediction by measuring the entropy [34], [11] or probability of class [10].”

“As of now, YMIR uses Label Studio [6] as the labeling op, adopts YOLOv4 [3] in DarkNet [5] as the training op and the data scoring methods in [1] as the core of our mining op.”

“Extended to the entropy of posterior probabilities [5, 6], emsemble measure disagreement [7], label correlation [8] and predicted learning loss [9] were further introduced as the estimated uncertainty for active data selection.”

“Note that, in this case, we can easily avoid to introduce too similar training samples coming from the same object instances in consecutive frames, which has shown to be useful in AL approaches [2].”

“For instance, one approach, known as active learning [33], [34], seeks to minimize manual annotation effort on a large dataset while maximizing the model’s performance [35], [36], [37]; Specifically, prior HITL methods for pose estimation [37], [38], [39] have involved actively selecting and…”

“Weakly-supervised Yuan et al. 2021; Aghdam et al. 2019; Choi et al. 2021), semantic segmentation (Kim et al. 2021; Siddiqui, Valentin, and Nießner 2020; Dai et al. 2020; Yang et al. 2017).”

“During the training process, f(Xi) are calculated for all (or part of) (Aghdam et al. 2019) the unlabeled samples U .”

“Query by uncertainty methods employed classification margin as a selection criteria [4], multi-class classification margin [24], learning loss [48], number of false-positive or false-negative pixels in images [2], discrepancy between two auxiliary classifiers [7].”

“While [15,1] aimed to create this uncertainty from the outputs by comparing an image and its noise version, [12,13] measured the stability of the prediction of the same image from different models’ inferences.”

“In object detection, Aghdam et al. (2019) assumed that the disagreement between probabilities in the neighborhood of wrongly predicted pixels should be high.”

“Recently, the active learning [3] has been applied to various deep learning tasks such as object detection [4], [9], person re-identification [10], multi-task learning [11], named entity recognition [12], human pose estimation [13], action localization [14], and biomedical image analysis [15], [16].”

“Other uncertainty-based methods attempt to aggregate pixel-level scores for each image [57], reformulate detectors by adding Bayesian inference to estimate the uncertainty [58] or replace conventional detection head with the Gaussian mixture model to compute aleatoric and epistemic uncertainty [13].”

“Also, pixel-level uncertainty can aggregate image-level uncertainty, according to the image-level uncertainty to select the most informative image [61].”

“Also, there are very few works of deep active learning for object detection [29], [40], [41].”