The DOI matches its DataCite record; 4 of the 4 references it cites were found in a registry or an index.
Understanding Intra-Class Knowledge Inside CNN
Open accessAbstract
Convolutional Neural Network (CNN) has been successful in image recognition tasks, and recent works shed lights on how CNN separates different classes with the learned inter-class knowledge through visualization. In this work, we instead visualize the intra-class knowledge inside CNN to better understand how an object class is represented in the fully-connected layers. To invert the intra-class knowledge into more interpretable images, we propose a non-parametric patch prior upon previous CNN visualization models. With it, we show how different "styles" of templates for an object class are organized by CNN in terms of location and content, and represented in a hierarchical and ensemble way. Moreover, such intra-class knowledge can be used in many interesting applications, e.g. style-based image retrieval and style-based object completion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Single neurons are polysemantic.
“However, researchers discovered that single neurons are 'polysemantic' -they seem to respond to multiple kinds of features in both artificial (Wei et al., 2015;Nguyen et al., 2016c;Olah et al., 2017a) and biological networks (Churchland & Shenoy, 2007;Rigotti et al., 2013;Mante et al., 2013;Raposo et al., 2014).”
From Open Problems in Mechanistic Interpretability (2025) · cites this paper as Wei et al., 2015
The Introduction (page unavailable) says individual CNN neurons can correspond to texture patterns or object detectors, while intra-class knowledge is captured collectively by groups of neurons. Neither that passage nor the passages on neural pathways and class variation say that a single neuron responds to multiple kinds of features. I searched for single-neuron responses to multiple features and for polysemanticity. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Flagged first, then by citations
Research works printed but not linked: 9
- P. Agrawal, R. Girshick, and J. Malik. Analyzing the perfor- mance of multilayer neural networks for object recognition. In Computer Vision-ECCV 2014, pages 329-344. Springer, 2014. 1, 4
- P. Baldi and P. J. Sadowski. Understanding dropout. In Advances in Neural Information Processing Systems, pages 2814-2822, 2013. 1
- C. Barnes, E. Shechtman, A. Finkelstein, and D. Goldman. Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics- TOG, 28(3):24, 2009. 5
- J. L. Long, N. Zhang, and T. Darrell. Do convnets learn cor- respondence? In Advances in Neural Information Processing Systems, pages 1601-1609, 2014. 3, 5, 6
- A. Mahendran and A. Vedaldi. Understanding deep image representations by inverting them. CVPR, 2015. 1, 2, 3, 5, 6
- J. L. P. Isola and E. Adelson. Discovering states and trans- formations in image collections. CVPR, 2015. 2
- C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013. 1 · arXiv
- M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In ECCV, pages 818-833. Springer, 2014. 1, 2
- B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Object detectors emerge in deep scene cnns. ICLR, 2014. 1, 3
Cited by
Sorted by citations
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167
“When visualizing convolutional neural networks trained on image data, work has been done on visualizing filters (62, 63), maximally activating responses of individual neurons or classes (64), understanding intra-class variation (65), and grouping different neurons (66).”
MethodOlah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
A Survey of Data-driven and Knowledge-aware eXplainable AI
Li, X.-H., Cao, C. C., Shi, Y., Bai, W., Gao, H., Qiu, L., Wang, C., Gao, Y., Zhang, S., Xue, X., et al. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2983930 · Cited by 224
Metadata
- Authors
- Wei, D., Zhou, B., Torrabla, A., & Freeman, W.
- Affiliations
- Massachusetts Institute of Technology (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1507.02379
- OpenAlex
- W830575572
- Cited by
- 98
- References
- 4
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 10 · 2023: 18 · 2022: 6 · 2021: 13 · 2020: 5 · 2019: 15 · 2018: 8 · 2017: 8 · 2016: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- convolutional neural networks, fully connected layers, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
