Self Paced Deep Learning for Weakly Supervised Object Detection
Open accessThe DOI matches its Crossref record; 48 of the 49 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
In a weakly-supervised scenario object detectors need to be trained using image-level annotation alone. Since bounding-box-level ground truth is not available, most of the solutions proposed so far are based on an iterative, Multiple Instance Learning framework in which the current classifier is used to select the highest-confidence boxes in each image, which are treated as pseudo-ground truth in the next training iteration. However, the errors of an immature classifier can make the process drift, usually introducing many of false positives in the training dataset. To alleviate this problem, we propose in this paper a training protocol based on the self-paced learning paradigm. The main idea is to iteratively select a subset of images and boxes that are the most reliable, and use them for training. While in the past few years similar strategies have been adopted for SVMs and other classifiers, we are the first showing that a self-paced approach can be used with deep-network-based classifiers in an end-to-end training pipeline. The method we propose is built on the fully-supervised Fast-RCNN architecture and can be applied to similar architectures which represent the input image as a bag of boxes. We show state-of-the-art results on Pascal VOC 2007, Pascal VOC 2010 and ILSVRC 2013. On ILSVRC 2013 our results based on a low-capacity AlexNet network outperform even those weakly-supervised approaches which are based on much higher-capacity networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Multi-instance learning is a group of supervised learning algorithms that has seen widespread application in WSOD [202-209].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [208]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“The ILSVRC 2013 detection dataset [38] is a standard benchmark for object detection.”
MethodRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“, hard-negative mining) can be useful in a supervised scenario [18], [19].”
BackgroundSelective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 2×
“A pioneering work in this direction is the curriculum learning approach proposed in [11].”
BackgroundReturn of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Training Region-Based Object Detectors with Online Hard Example Mining
Shrivastava, A., Gupta, A., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.89 · Cited by 2,686 · cited 2×
“The supervised state-of-the-art object-detection approaches [12], [18], [34] on Pascal VOC [35], [36] are based on the FastRCNNarchitecture,whosemain characteristic is theRoI pooling layer.”
BackgroundMethodThe PASCAL visual object classes challenge 2006 (VOC2006) results
Everingham, M., Zisserman, A., Williams, C. K. I., & Gool, L. V. (2006) · Cited by 2,223
Self-Paced Learning for Latent Variable Models
Kumar, M., Packer, B., & Koller, D. (2010) · neural information processing systems · Cited by 1,708
Is object localization for free? - Weakly-supervised learning with convolutional neural networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298668 · Cited by 886
Ensemble of exemplar-SVMs for object detection and beyond
Malisiewicz, T., Gupta, A., & Efros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126229 · Cited by 865 · cited 1×
“[7] use Exemplar SVMs (ESVMs) [26] to train a classifier from a single positive sample.”
MethodReturn of the Devil in the Details: Delving Deep into Convolutional NetsPreprint
Chatfield, K., Simonyan, K., Vedaldi, A., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3531 · Cited by 663
STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation
Wei, Y., Liang, X., Chen, Y., Shen, X., Cheng, M.-M., Feng, J., Zhao, Y., & Yan, S. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2636150 · Cited by 576
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2014) · arXiv · doi:10.48550/arxiv.1409.0575 · Cited by 567
Undoing the Damage of Dataset Bias
Khosla, A., Zhou, T., Malisiewicz, T., Efros, A. A., & Torralba, A. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_12 · Cited by 480
Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning
Cinbis, R. G., Verbeek, J., & Schmid, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2535231 · Cited by 420
Self-Paced Learning with Diversity
Jiang, L., Meng, D., Yu, S.-I., Lan, Z., Shan, S., & Hauptmann, A. G. (2014) · neural information processing systems · Cited by 410
Contextual Action Recognition with R* CNN
Gkioxari, G., Girshick, R., & Malik, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.129 · Cited by 356
Webly Supervised Learning of Convolutional Networks
Chen, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.168 · Cited by 351
Self-Paced Learning for Long-Term Tracking
Supancic III, J. S., & Ramanan, D. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.308 · Cited by 290
LSDA: Large Scale Detection Through Adaptation
Hoffman, J., Guadarrama, S., Tzeng, E., Hu, R., Donahue, J., Girshick, R., Darrell, T., & Saenko, K. (2014) · arXiv · doi:10.48550/arxiv.1407.5035 · Cited by 284 · cited 6×
“, [16], [29], [30]) object detection experiments.”
BackgroundMethodInfluentialZaremba, W., & Sutskever, I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.4615 · Cited by 283
ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization
Kantorov, V., Oquab, M., Cho, M., & Laptev, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_22 · Cited by 266
Multi-fold MIL Training for Weakly Supervised Object Localization
Cinbis, R. G., Verbeek, J., & Schmid, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.309 · Cited by 219
Curriculum learning of multiple tasks
Pentina, A., Sharmanska, V., & Lampert, C. H. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299188 · Cited by 213
On learning to localize objects with minimal supervisionPreprint
Song, H. O., Girshick, R., Jegelka, S., Mairal, J., Harchaoui, Z., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.1024 · Cited by 208
Weakly supervised object detection with convex clustering
Bilen, H., Pedersoli, M., & Tuytelaars, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298711 · Cited by 206
Weakly Supervised Object Localization with Progressive Domain Adaptation
Li, D., Huang, J.-B., Li, Y., Wang, S., & Yang, M.-H. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.382 · Cited by 201
Learning the easy things first: Self-paced visual category discovery
Lee, Y. J., & Grauman, K. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995523 · Cited by 168
Weakly Supervised Deep Detection Networks
Bilen, H., & Vedaldi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.311 · Cited by 144
Self-taught object localization with deep networks
Bazzani, L., Bergamo, A., Anguelov, D., & Torresani, L. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477688 · Cited by 140
Weakly Supervised Object Detection with Posterior Regularization
Bilen, H., Pedersoli, M., & Tuytelaars, T. (2014) · Lirias · Cited by 100
Weakly-supervised Discovery of Visual Pattern Configurations
Song, H. O., Lee, Y. J., Jegelka, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6507 · Cited by 94
Towards Computational Baby Learning: A Weakly-Supervised Approach for Object Detection
Liang, X., Liu, S., Wei, Y., Liu, L., Lin, L., & Yan, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.120 · Cited by 88
Are all training examples equally valuablePreprint
Lapedriza, À., Pirsiavash, H., Bylinskii, Z., & Torralba, A. B. (2013) · arXiv (Cornell University) · doi:10.13016/m2jp7z-giib · Cited by 84
Zhang, D., Meng, D., Zhao, L., & Han, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01290 · Cited by 79
Detector discovery in the wild: Joint multiple instance and representation learning
Hoffman, J., Pathak, D., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298906 · Cited by 75
Attention Networks for Weakly Supervised Object Localization
Teh, E., Rochan, M., & Wang, Y. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.52 · Cited by 70
Weakly Supervised Object Localization Using Size Estimates
Shi, M., & Ferrari, V. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_7 · Cited by 67
Large-Scale Weakly Supervised Object Localization via Latent Category Learning
Wang, C., Huang, K., Ren, W., Zhang, J., & Maybank, S. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2396361 · Cited by 54
Learning discriminative localization from weakly labeled data
Hoai, M., Torresani, L., la Torre, F. D., & Rother, C. (2013) · Pattern Recognition · doi:10.1016/j.patcog.2013.09.028 · Cited by 39
Computational Baby LearningPreprint
Liang, X., Liu, S., Wei, Y., Liu, L., Lin, L., & Yan, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2861 · Cited by 17
Statistical and Spatial Consensus Collection for Detector Adaptation
Sangineto, E. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_30 · Cited by 6
Self-citation
Cited by
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689
“Multi-instance learning is a group of supervised learning algorithms that has seen widespread application in WSOD [202], [203], [204], [205], [206], [207], [208], [209].”
Convolutional neural network: a review of models, methodologies and applications to object detection
Dhillon, A., & Verma, G. K. (2020) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00203-0 · Cited by 1,119
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
Jiang, L., Zhou, Z., Leung, T., Li, L.-J., & Fei-Fei, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05055 · Cited by 503
Soviany, P., Ionescu, R. T., Rota, P., & Sebe, N. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01611-x · Cited by 431
Self-citationZhou, X., Xu, X., Liang, W., Zeng, Z., Shimizu, S., Yang, L. T., & Jin, Q. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3061419 · Cited by 308
Metadata
- Authors
- Sangineto, E., Nabi, M., Culibrk, D., & Sebe, N.
- Affiliations
- University of Trento (IT)
- Systems, Applications & Products in Data Processing (Germany) (DE)
- University of Novi Sad (RS)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-03-01
- Volume
- 41
- Issue
- 3
- Pages
- 712-725
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2494236530
- Cited by
- 114
- References
- 49
- Citations by year
- 2025: 6 · 2024: 9 · 2023: 12 · 2022: 14 · 2021: 16 · 2020: 21 · 2019: 25
- FWCI
- 8.63
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Iris (University of Trento) (submittedVersion, other-oa)
- IRIS UNIMORE (University of Modena and Reggio Emilia) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Nvidia
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- weakly supervised object detection, self-paced learning, image-level annotations, multiple instance learning, pseudo-labeling, training data selection, Fast R-CNN, AlexNet, Pascal VOC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.