The DOI matches its DataCite record; 5 of the 5 references it cites were found in a registry or an index.
Object Detectors Emerge in Deep Scene CNNs
Open accessAbstract
With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly. One important factor for continued progress is to understand the representations that are learned by the inner layers of these deep architectures. Here we show that object detectors emerge from training CNNs to perform scene classification. As scenes are composed of objects, the CNN for scene classification automatically discovers meaningful objects detectors, representative of the learned scene categories. With object detectors emerging as a result of learning to recognize scenes, our work demonstrates that the same network can perform both scene recognition and object localization in a single forward-pass, without ever having been explicitly taught the notion of objects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many models form at least some interpretable features.
“Many models form at least some interpretable features.”
From Toy Models of Superposition (2022) · cites this paper as [13]
Section 3.3 reports interpretable units in both the Places-CNN and ImageNet-CNN, including units responsive to objects and scenes. That supports the point for the two networks studied, but not the broader claim about many models. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThere is a significant body of results finding neurons which appear to be interpretable in RNNs, CNNs, and GANs, activating in response to some understandable property.
“Interpretable Neurons - There is a significant body of results finding neurons which appear to be interpretable ( in RNNs ; in CNNs ; in GANs ), activating in response to some understandable property.”
From Toy Models of Superposition (2022) · cites this paper as [13]
There is evidence of interpretable neurons in RNNs, convolutional neural networks, and in some limited cases, transformer language models.
“There is evidence of interpretable neurons in RNNs ( e.g. ), convolutional neural networks ( see generally e.g. ; individual neuron families ), and in some limited cases, transformer language models ( see detailed discussion in our previous paper ).”
From Toy Models of Superposition (2022) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
Flagged first, then by citations
Research works printed but not linked: 16
- Agrawal, Pulkit, Girshick, Ross, and Malik, Jitendra. Analyzing the performance of multilayer neural networks for object recognition. ECCV, 2014.
- Bergamo, Alessandro, Bazzani, Loris, Anguelov, Dragomir, and Torresani, Lorenzo. Self-taught object localization with deep networks. arXiv preprint arXiv:1409.3964, 2014. · arXiv
- Biederman, Irving. Visual object recognition, volume 2. MIT press Cambridge, 1995.
- Lin, Tg-Yi, Maire, Michael, Belongie, Serge, Hays, James, Perona, Pietro, Ramanan, Deva, Dollr, Piotr, and Zitnick, C. Lawrence. Microsoft COCO: Common objects in context. In ECCV, 2014.
- Long, Jonathan, Zhang, Ning, and Darrell, Trevor. Do convnets learn correspondence? In NIPS, 2014.
- Oquab, Maxime, Bottou, Léon, Laptev, Ivan, Sivic, Josef, et al. Weakly supervised object recogni- tion with convolutional neural networks. In NIPS. 2014.
- Pandey, M. and Lazebnik, S. Scene recognition and weakly supervised object localization with deformable part-based models. In ICCV, 2011.
- Pérez, Patrick, Gangnet, Michel, and Blake, Andrew. Poisson image editing. ACM Trans. Graph., 2003.
- Renninger, Laura Walker and Malik, Jitendra. When is scene identification just texture recognition? Vision research, 44(19):2301-2311, 2004.
- Szegedy, Christian, Zaremba, Wojciech, Sutskever, Ilya, Bruna, Joan, Erhan, Dumitru, Goodfellow, Ian, and Fergus, Rob. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013. · arXiv
- Tanaka, Keiji. Neuronal mechanisms of object recognition. Science, 262(5134):685-688, 1993.
- Tang, Yichuan, Srivastava, Nitish, and Salakhutdinov, Ruslan R. Learning generative models with visual attention. In NIPS. 2014.
- Xiao, J, Ehinger, K A., Hays, J, Torralba, A, and Oliva, A. SUN database: Exploring a large collection of scene categories. IJCV, 2014.
- Yosinski, Jason, Clune, Jeff, Bengio, Yoshua, and Lipson, Hod. How transferable are features in deep neural networks? In NIPS, 2014.
- Zeiler, M. and Fergus, R. Visualizing and understanding convolutional networks. In ECCV, 2014.
- Zhou, Bolei, Lapedriza, Agata, Xiao, Jianxiong, Torralba, Antonio, and Oliva, Aude. Learning deep features for scene recognition using places database. In NIPS, 2014.
Cited by
Sorted by citations
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Self-citationBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965
Unified Perceptual Parsing for Scene Understanding
Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_26 · Cited by 2,201
“To overcome the issue raised by Zhou et al. [32] that although the theoretical receptive field of deep CNN is large enough, the empirical receptive field of deep CNN is relatively much smaller [33], we apply a Pyramid Pooling Module (PPM) from PSPNet [16] on the last layer of the backbone network…”
MethodSelf-citationLarge Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network
Peng, C., Zhang, X., Yu, G., Luo, G., & Sun, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.189 · Cited by 1,750
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360
Kriegeskorte, N. (2015) · Annual Review of Vision Science · doi:10.1146/annurev-vision-082114-035447 · Cited by 1,093
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097
Pros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958
Is object localization for free? - Weakly-supervised learning with convolutional neural networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298668 · Cited by 886
Interpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833
Yang, F., Choi, W., & Lin, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.234 · Cited by 608
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Dong, H., Song, K., He, Y., Xu, J., Yan, Y., & Meng, Q. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2958826 · Cited by 559
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., & Rohrbach, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00915 · Cited by 418
Exploring Categorical Regularization for Domain Adaptive Object Detection
Xu, C.-D., Zhao, X.-R., Jin, X., & Wei, X.-S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01174 · Cited by 355
Learning to Generate Reviews and Discovering Sentiment
Radford, A., Jozefowicz, R., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1704.01444 · Cited by 330
“Analysis of the task specific representations learned by these models reveals many fascinating properties (Zhou et al., 2014).”
BackgroundComputing receptive fields of convolutional neural networks
Araujo, A., Norris, W., & Sim, J. (2019) · Distill · doi:10.23915/distill.00021 · Cited by 312
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
Zoom In: An Introduction to Circuits
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020) · Distill · doi:10.23915/distill.00024.001 · Cited by 288
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2015) · arXiv · doi:10.48550/arxiv.1512.02325 · Cited by 22,443
“Feature maps from different levels within a network are known to have different (empirical) receptive field sizes [24].”
Background[Paper] Visual Instance Retrieval with Deep Convolutional Networks
Razavian, A. S., Sullivan, J., Carlsson, S., & Maki, A. (2016) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.4.251 · Cited by 276
Learning Aligned Cross-Modal Representations from Weakly Aligned Data
Castrejon, L., Aytar, Y., Vondrick, C., Pirsiavash, H., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.321 · Cited by 188
Self-citationSoft Proposal Networks for Weakly Supervised Object Localization
Zhu, Y., Zhou, Y., Ye, Q., Qiu, Q., & Jiao, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.204 · Cited by 156
Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks
Wagner, J., Köhler, J. M., Gindele, T., Hetzel, L., Wiedemer, J. T., & Behnke, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00931 · Cited by 144
Self-taught object localization with deep networks
Bazzani, L., Bergamo, A., Anguelov, D., & Torresani, L. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477688 · Cited by 140
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122
Demystifying Brain Tumor Segmentation Networks: Interpretability and Uncertainty Analysis
Natekar, P., Kori, A., & Krishnamurthi, G. (2020) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.00006 · Cited by 103
Kortylewski, A., Liu, Q., Wang, A., Sun, Y., & Yuille, A. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01401-3 · Cited by 98
Computational mechanisms underlying cortical responses to the affordance properties of visual scenes
Bonner, M. F., & Epstein, R. A. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006111 · Cited by 94
Weakly Supervised Deep Detection Networks
Bilen, H., & Vedaldi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.311 · Cited by 144
“For example, there exists evidence that CNNs trained for image classification learn proxies to objects and objects parts [37].”
BackgroundExplaining Neural Networks Semantically and Quantitatively
Chen, R., Chen, H., Huang, G., Ren, J., & Zhang, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00928 · Cited by 56
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.10652 · Cited by 51
A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks
Grün, F., Rupprecht, C., Navab, N., & Tombari, F. (2016) · arXiv · doi:10.48550/arxiv.1606.07757 · Cited by 49
Izadyyazdanabadi, M., Belykh, E., Mooney, M. A., Eschbacher, J. M., Nakaji, P., Yang, Y., & Preul, M. C. (2018) · Frontiers in Oncology · doi:10.3389/fonc.2018.00240 · Cited by 45
Gao, J., Yang, J., Wang, G., & Li, M. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.06.055 · Cited by 44
Automated ultrasound assessment of amniotic fluid index using deep learning
Cho, H. C., Sun, S., Min Hyun, C., Kwon, J.-Y., Kim, B., Park, Y., & Seo, J. K. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101951 · Cited by 43
Metadata
- Authors
- Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.6856
- OpenAlex
- W1899185266
- Cited by
- 971
- References
- 5
- Citations by year
- 2026: 11 · 2025: 37 · 2024: 53 · 2023: 85 · 2022: 83 · 2021: 128 · 2020: 135 · 2019: 135 · 2018: 122 · 2017: 103 · 2016: 59 · 2015: 20
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1016862)
- Nvidia
- Multidisciplinary University Research Initiative (N00014-10-1-0933)
- Office of Naval Research (N00014-10-1-0933)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object detection, scene classification, convolutional neural networks, object localization, scene understanding, learned representations, single forward pass
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
