Self-taught object localization with deep networks
The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
This paper introduces self-taught object localization, a novel approach that leverages deep convolutional networks trained for whole-image recognition to localize objects in images without additional human supervision, i.e., without using any ground-truth bounding boxes for training. The key idea is to analyze the change in the recognition scores when artificially masking out different regions of the image. The masking out of a region that includes the object typically causes a significant drop in recognition score. This idea is embedded into an agglomerative clustering technique that generates self-taught localization hypotheses. Our object localization scheme outperforms existing proposal methods in both precision and recall for small number of subwindow proposals (e.g., on ILSVRC-2012 it produces a relative gain of 23.4% over the state-of-the-art for top-1 hypothesis). Furthermore, our experiments show that the annotations automatically-generated by our method can be used to train object detectors yielding recognition results remarkably close to those obtained by training on manually-annotated bounding boxes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Some other researchers proposed to mask out different parts of the image. If the detection score drops sharply, then the masked region may contain an object with high probability [214].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [214]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Efficient Graph-Based Image Segmentation
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000022288.19776.77 · Cited by 6,176
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 2×
“In order to better comprehend deep networks, previous work proposed to visualize the intermediate representations [33, 28], give semantic interpretation of individual units [19], study the emergence of detectors [35] or fool them with artificial images [22].”
BackgroundMethodDeep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258
Research works printed but not linked: 2
- everingham, 0, The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results
- 2012, Large Scale Visual Recognition Challenge
Cited by
Items 101–145 of 145
Weakly Supervised Object Detection for Aerial Images With Instance-Aware Label Assignment
Xie, X., Yuan, X., & Cheng, G. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3609322 · Cited by 5
A Survey on Tools and Techniques for Localizing Abnormalities in X-ray Images Using Deep Learning
Aasem, M., Iqbal, M. J., Ahmad, I. A., Alassafi, M. O., & Alhomoud, A. (2022) · Mathematics · doi:10.3390/math10244765 · Cited by 8
RETRACTED: Sports highlight recognition and event detection using rule inference system
Soundararajan, K., & Mala, T. (2022) · Concurrent Engineering · doi:10.1177/1063293x221088353 · Cited by 5
RetractedKnowledge Distillation with Category-Aware Attention and Discriminant Logit Losses
Jiang, L., Zhou, W., & Li, H. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00308 · Cited by 5
Real-Time Object Detection and Localization for Vision-Based Robot Manipulator
Batra, V., & Kumar, V. (2021) · SN Computer Science · doi:10.1007/s42979-021-00561-4 · Cited by 5
Attaining Real-Time Super-Resolution for Microscopic Images Using GANPreprint
Bhatia, V., & Kumar, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04634 · Cited by 5
Attribute saliency network for person re-identification
Tay, C.-P., & Yap, K. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104298 · Cited by 4
Tilgner, S., Wagner, D., Kalischewski, K., Schmitz, J.-C., & Kummert, A. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287315 · Cited by 5
Feature‐Enhanced Occlusion Perception Object Detection for Smart Cities
Xu, J., Wang, H., Xu, M., Yang, F., Zhou, Y., & Yang, X. (2021) · Wireless Communications and Mobile Computing · doi:10.1155/2021/5544194 · Cited by 4
Weakly Supervised PET Tumor Detection Using Class ResponsePreprint
Amyar, A., Modzelewski, R., Véra, P., Morard, V., & Ruan, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08337 · Cited by 5
Segmentation and Shape Extraction from Convolutional Neural Networks
Ha, M. L., Franchi, G., Möller, M., Kolb, A., & Blanz, V. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00169 · Cited by 5
Multimodal Fused Deep Learning Networks for Domain Specific Image Similarity Search
Waqas, U., Visser, J. W., Choe, H., & Lee, D. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.035716 · Cited by 3
Interactive machine vision for wildlife conservation
Kellenberger, B. (2020) · doi:10.18174/511122 · Cited by 3
Hide-CAM: Finding Multiple Discriminative Regions in Weakly Supervised Location
Xu, J., Sheng, S., Wei, H., & Guo, J. (2019) · IEEE Access · doi:10.1109/access.2019.2939267 · Cited by 2
Model Interpretability and Rationale Extraction by Input Mask Optimization
Brinner, M., & Zarrieß, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.867 · Cited by 8
Seg-Cam: Enhancing interpretability analysis in segmentation networks
Wu, W., Ye, C., & Liao, X. (2025) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2025.104467 · Cited by 2
Cheng, Z., Li, H., Zeng, X., Wang, M., & Duan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09556-4 · Cited by 2
A Comprehensive Analysis of Methodologies used for Object Detection and Localization
Delight, D. T., & Karunakaran, V. (2021) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs51430.2021.9441916 · Cited by 2
Interpreting and Comparing Convolutional Neural Networks: A Quantitative Approach
Islam, M. M., & Tushar, Z. H. (2021) · International Conference on Electrical Engineering and Information Communication Technology · doi:10.1109/iceeict53905.2021.9667854 · Cited by 2
Sound Event Detection on GradCAM method with Complex Pretrained Model
Cai, X., Qiao, Z., Gan, Y., Wu, J., Wu, M., & Dong, H. (2022) · International Conference on Computer Graphics, Image and Virtualization (ICCGIV) · doi:10.1109/iccgiv57403.2022.00024 · Cited by 4
Machine Learning—Basic Unsupervised Methods (Cluster Analysis Methods, t-SNE)
Espadoto, M., Martins, S. B., Branderhorst, W., & Telea, A. C. (2023) · Springer eBooks · doi:10.1007/978-3-031-36678-9_9 · Cited by 1
Component Awareness in Convolutional Neural Networks
Iwana, B. K., Zhou, L., Tanaka‐Ishii, K., & Uchida, S. (2017) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2017.72 · Cited by 1
RM-NET: Rasterizing Markov Signals to Images for Deep Learning
Gupta, K., & Wei, Z. (2023) · International Conference on Cyber-Physical Social Intelligence (ICCSI) · doi:10.1109/iccsi58851.2023.10303906 · Cited by 1
Learning a Weight Map for Weakly-Supervised Localization
Shaharabany, T., & Wolf, L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094940 · Cited by 1
Points2Shapelets: A Salience-Guided Shapelets Selection Approach to Time Series Classification
Feng, G., Ma, C., Zhou, L., Wu, L., Zhang, J., & Shi, X. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892079 · Cited by 1
Next generation analysis of earthquakes and earthquake catalogs with InSAR
Brengman, C. M. J., Barnhart, W., Yeck, W. L., Reagan, M., McClelland, B., & Finzel, E. (2021) · doi:10.17077/etd.006012 · Cited by 1
A Swin-Transformer Based Multi-Scale Context Fusion Network for Coal Gangue SegmentationPreprint
Wang, Z., Zhang, Z., Xu, S., & Leng, Z. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4591254 · Cited by 1
Model assisted bootstrapping for annotation of segmentation datasets
Batchelor, O., & Green, R. R. (2017) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz.2017.8402450
Artifact Detection Maps Learned using Shallow Convolutional Networks
Goodall, T., & Bovik, A. C. (2018) · IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI) · doi:10.1109/ssiai.2018.8470369
A Classify-Before-Detect Method For Weakly Supervised Object Localization
Dong, X., Wang, P., Fu, R., Gao, Y., & Wang, J. (2019) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1176/3/032003
Weakly Supervised Object Localization with Inter-Intra Regulated CAMs.Preprint
Cui, G., Kou, Z., Wang, S., Zhao, W., & Xu, C. (2019) · arXiv (Cornell University)
Target Cropping: A New Data Augmentation Method of Fine-Grained Image Classification
Lu, J., & Liao, M. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-32456-8_37
Research on Automatic Wharf Unmanned Gate System Based on Artificial Intelligence
Jie-min, Y., Ding, X., Li, Y., & Lin, J. (2019) · International Conference on Information Technology: IoT and Smart City (ICIT) · doi:10.1145/3377170.3377280
MakeupBag: Disentangling Makeup Extraction and ApplicationPreprint
Hoshen, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02157 · Cited by 1
Interpreting and Comparing Convolutional Neural Networks: A Quantitative ApproachPreprint
Islam, M. M., & Tushar, Z. H. (2021) · Preprints.org · doi:10.20944/preprints202101.0579.v1
Weakly Supervised Volumetric Segmentation via Self-taught Shape Denoising ModelPreprint
He, Q., Li, S., & He, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13082 · Cited by 2
Video Abnormal Event Detection and Location Based on Spatial Attention
Guo, Z., Wu, G., & Li, L. (2021) · International Conference on Graphics and Signal Processing (ICGSP) · doi:10.1145/3474906.3474917
Interpreting and Comparing Convolutional Neural Networks: A Quantitative ApproachPreprint
Islam, M. M., & Tushar, Z. H. (2021) · Preprints.org · doi:10.20944/preprints202101.0579.v2
Tankala, M. R., Stella, S., Sandepogu, P., Gopal, K. N., Mamillapalli, R. B., & Rambabu, D. (2022) · Advances in computational intelligence and robotics book series · doi:10.4018/978-1-6684-3981-4.ch014
Learning from limited labelled data: contributions to weak, few-shot, and unsupervised learning
Rodríguez, J. J. S. (2022) · RiuNet (Universitat Politècnica de València) · doi:10.4995/thesis/10251/190633
Nhận dạng một số bệnh trên bông xoài sử dụng kỹ thuật học sâu chuyển tiếp
Phan, T. P., Ngo, Q. H., & Trương, Q. B. (2024) · Can Tho University Journal of Science · doi:10.22144/ctujos.2024.237
Enhancing medical AI interpretability using heatmap visualization techniques
Moeini, E., Çakır, D., & Arıca, N. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-23979-3.00004-x · Cited by 1
Acremont, A. D. (2020) · theses.fr (ABES) · doi:10.70675/8760fbeez9087z45a7z8958z74f77c24d5e4 · Cited by 1
Apprentissage profond pour l'analyse de la qualité des pommes de terre
Marino, S. (2020) · theses.fr (ABES) · doi:10.70675/4ee3e954z1d5bz4a17zb3e8z862468003d1f
Annotation-efficient learning for object discovery and detection
Vo, H. V. (2022) · theses.fr (ABES) · doi:10.70675/aae8cd95z8e92z4d35z9384zafe56efdad87
Metadata
- Authors
- Bazzani, L., Bergamo, A., Anguelov, D., & Torresani, L.
- Affiliations
- Dartmouth College (US)
- Google (United States) (US)
- ORCID
- Torresani, Lorenzo
- Venue
- 2016 IEEE Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2016-03
- Pages
- 1-9
- ISSN
- 2472-6737 (ISSN-L)
- OpenAlex
- W1586079445
- Cited by
- 140
- References
- 39
- Citations by year
- 2025: 4 · 2024: 3 · 2023: 13 · 2022: 17 · 2021: 23 · 2020: 19 · 2019: 21 · 2018: 15 · 2017: 16 · 2016: 5
- FWCI
- 6.59
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Winter Conference on Applications of Computer Vision (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- agglomerative clustering, bounding box annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.