The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Deep multiple instance learning for image classification and auto-annotation
Abstract
The recent development in learning deep representations has demonstrated its wide applications in traditional vision tasks like classification and detection. However, there has been little investigation on how we could build up a deep learning framework in a weakly supervised setting. In this paper, we attempt to model deep learning in a weakly supervised learning (multiple instance learning) framework. In our setting, each image follows a dual multi-instance assumption, where its object proposals and possible text annotations can be regarded as two instance sets. We thus design effective systems to exploit the MIL property with deep learning strategies from the two ends; we also try to jointly learn the relationship between object and annotation proposals. We conduct extensive experiments and prove that our weakly supervised deep learning framework not only achieves convincing performance in vision tasks including classification and image annotation, but also extracts reasonable region-keyword pairs with little supervision, on both widely used benchmarks like PASCAL VOC and MIT Indoor Scene 67, and also a dataset for image-and patch-level annotations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Saliency Detection: A Spectral Residual Approach
Hou, X., & Zhang, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383267 · Cited by 3,422
Solving the multiple instance problem with axis-parallel rectangles
Dietterich, T. G., Lathrop, R. H., & Lozano-Pérez, T. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00034-3 · Cited by 2,866
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
The PASCAL visual object classes challenge 2006 (VOC2006) results
Everingham, M., Zisserman, A., Williams, C. K. I., & Gool, L. V. (2006) · Cited by 2,079
Learning to Detect a Salient Object
Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., & Shum, H.-Y. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.70 · Cited by 1,777
A survey of content-based image retrieval with high-level semantics
Liu, Y., Zhang, D., Lu, G., & Ma, W.-Y. (2007) · Pattern Recognition · doi:10.1016/j.patcog.2006.04.045 · Cited by 1,696 · cited 1×
“Image annotation aims at producing rich image descriptions at different levels [26].”
BackgroundQuattoni, A., & Torralba, A. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206537 · Cited by 1,463
Support Vector Machines for Multiple-Instance Learning
Andrews, S. J., Tsochantaridis, I., & Hofmann, T. (2002) · neural information processing systems · Cited by 1,401
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Cheng, M.-M., Zhang, Z., Lin, W.-Y., & Torr, P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.414 · Cited by 1,050
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Zhuowen Tu, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 993
Learning Algorithms for Keyphrase Extraction
Turney, P. D. (2000) · Information Retrieval · doi:10.1023/a:1009976227802 · Cited by 889
Deeplysupervised netsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015) · Cited by 849
Li, L.-J., Su, H., Fei-Fei, L., & Xing, E. P. (2018) · KiltHub Repository · doi:10.1184/r1/6475985 · Cited by 827
What, where and who? Classifying events by scene and object recognition
Li, L.-J., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408872 · Cited by 781
Multiple Instance Boosting for Object Detection
Zhang, C., Platt, J. C., & Viola, P. A. (2005) · neural information processing systems · Cited by 685
Using Multiple Segmentations to Discover Objects and their Extent in Image Collections
Russell, B. C., Freeman, W. T., Efros, A. A., Sivic, J., & Zisserman, A. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.326 · Cited by 617
Real-Time Computerized Annotation of Pictures
Li, J., & Wang, J. Z. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70847 · Cited by 506
Learning Convolutional Feature Hierarchies for Visual Recognition
Kavukcuoglu, K., Sermanet, P., Boureau, Y.-L., Gregor, K., Mathieu, M., & LeCun, Y. (2010) · neural information processing systems · Cited by 485
SuperParsing: Scalable Nonparametric Image Parsing with Superpixels
Tighe, J., & Lazebnik, S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_26 · Cited by 477
Barnard, K., Duygulu, P., Forsyth, D. A., de Freitas, N., Blei, D. M., Jordan, M. I., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2003) · Cited by 459
Harvey, D., & Moran, S. (2009) · Cited by 439
Scene recognition and weakly supervised object localization with deformable part-based models
Pandey, M., & Lazebnik, S. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126383 · Cited by 408
Blocks That Shout: Distinctive Parts for Scene Classification
Juneja, M., Vedaldi, A., Jawahar, C. V., & Zisserman, A. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.124 · Cited by 368
Deep learning of feature representation with multiple instance learning for medical image analysis
Xu, Y., Mo, T., Feng, Q., Zhong, P., Lai, M., & Chang, E. I.-C. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853873 · Cited by 354
SUN Database: Exploring a Large Collection of Scene Categories
Xiao, J., Ehinger, K. A., Hays, J., Torralba, A., & Oliva, A. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-014-0748-y · Cited by 331
Mid-level Visual Element Discovery as Discriminative Mode Seeking
Doersch, C., Gupta, A., & Efros, A. A. (2013) · neural information processing systems · Cited by 238
Contextualizing object detection and classification
Song, Z., Chen, Q., Huang, Z., Hua, Y., & Yan, S. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995330 · Cited by 182
MILCut: A Sweeping Line Multiple Instance Learning Paradigm for Interactive Image Segmentation
Wu, J., Zhao, Y., Zhu, J.-Y., Luo, S., & Tu, Z. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.40 · Cited by 161
Self-citationWeakly supervised discriminative localization and classification: a joint learning process
Nguyen, M. H., Torresani, L., de la Torre, F., & Rother, C. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459426 · Cited by 159
Joint learning of visual attributes, object classes and visual saliency
Wang, G., & Forsyth, D. A. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459194 · Cited by 149
Neural Networks for Multi-Instance Learning
Zhou, Z., & Zhang, M.-L. (2002) · Cited by 146
Reconfigurable models for scene recognition
Parizi, S. N., Oberlin, J., & Felzenszwalb, P. F. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248001 · Cited by 144
Harvesting Mid-level Visual Concepts from Large-Scale Internet Images
Li, Q., Wu, J., & Tu, Z. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.115 · Cited by 141
Self-citationLearning Discriminative Part Detectors for Image Classification and Cosegmentation
Sun, J. Z., & Ponce, J. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.422 · Cited by 132
Keywords to visual categories: Multiple-instance learning forweakly supervised object categorization
Vijayanarasimhan, S., & Grauman, K. L. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587632 · Cited by 122
Text-based image retrieval using progressive multi-instance learning
Li, W., Duan, L., Xu, D., & Tsang, I. W.-H. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126478 · Cited by 120
Weakly-supervised Discovery of Visual Pattern ConfigurationsPreprint
Song, H. O., Lee, Y. J., Jegelka, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6507 · Cited by 110
Weakly Supervised Object Localization with Stable Segmentations
Galleguillos, C., Babenko, B., Rabinovich, A., & Belongie, S. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88682-2_16 · Cited by 95
Subcategory-Aware Object Classification
Dong, J., Xia, W., Chen, Q. K., Feng, J., Huang, Z., & Yan, S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.112 · Cited by 93
Hierarchical matching with side information for image classification
Chen, Q. K., Zheng, S., Yang, H., Huang, Z., & Yan, S. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248083 · Cited by 77
Multi-instance learning using recurrent neural networks
Garcez, A. D., & Zaverucha, G. (2012) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252784 · Cited by 14
Cited by
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793
Multiple instance learning: A survey of problem characteristics and applications
Carbonneau, M.-A., Cheplygina, V., Granger, E., & Gagnon, G. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.009 · Cited by 740
“The weak supervision can come from description sentences [103–105], web search engine results [106], tags associated with similar images and words found on web pages associated with the images [2].”
BackgroundLess is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
Lei, J., Li, L., Zhou, L., Gan, Z., Berg, T. L., Bansal, M., & Liu, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00725 · Cited by 589
Revisiting multiple instance neural networks
Wang, X., Yan, Y., Tang, P., Bai, X., & Liu, W. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.026 · Cited by 498
Metadata
- Authors
- Wu, J., Yu, Y., Huang, C., & Yu, K.
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3460-3469
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1938425378
- Cited by
- 386
- References
- 47
- FWCI
- 16.49
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.