The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Interpreting Deep Visual Representations via Network Dissection
Open accessAbstract
The success of recent deep convolutional neural networks (CNNs) depends on learning hidden representations that can summarize the important factors of variation behind the data. In this work, we describe Network Dissection, a method that interprets networks by providing meaningful labels to their individual units. The proposed method quantifies the interpretability of CNN representations by evaluating the alignment between individual hidden units and visual semantic concepts. By identifying the best alignments, units are given interpretable labels ranging from colors, materials, textures, parts, objects and scenes. The method reveals that deep representations are more transparent and interpretable than they would be under a random equivalently powerful basis. We apply our approach to interpret and compare the latent representations of several network architectures trained to solve a wide range of supervised and self-supervised tasks. We then examine factors affecting the network interpretability such as the number of the training iterations, regularizations, different initialization parameters, as well as networks depth and width. Finally we show that the interpreted units can be used to provide explicit explanations of a given CNN prediction for an image. Our results highlight that interpretability is an important property of deep neural networks that provides new insights into what hierarchical structures can learn.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Cited by
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Unified Perceptual Parsing for Scene Understanding
Xiao, T., Liu, Y., Zhou, B., Jiang, Y., & Sun, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_26 · Cited by 2,201
“Nevertheless, since it is originally collected to discover the alignment between visual concepts and hidden units of Convolutional Neural Networks (CNNs) for network interpretability [26,29], we find that samples from different classes are unbalanced.”
BackgroundSelf-citationExplaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
“Further methods do not seek to explain in terms of input features but in terms of the latent space, where the directions in the latent space code for higher level concepts, such as color, material, object part, or object [17], [18], [205].”
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing
Monga, V., Li, Y., & Eldar, Y. C. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.3016905 · Cited by 1,414
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958
A Survey on Neural Network Interpretability
Zhang, Y., Tino, P., Leonardis, A., & Tang, K. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3100641 · Cited by 923
Transformer Interpretability Beyond Attention Visualization
Chefer, H., Gur, S., & Wolf, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00084 · Cited by 903
Understanding the role of individual units in a deep neural network
Bau, D., Zhu, J.-Y., Strobelt, H., Lapedriza, A., Zhou, B., & Torralba, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907375117 · Cited by 375
Self-citationInterpreting Super-Resolution Networks with Local Attribution Maps
Gu, J., & Dong, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00908 · Cited by 361
Computing receptive fields of convolutional neural networks
Araujo, A., Norris, W., & Sim, J. (2019) · Distill · doi:10.23915/distill.00021 · Cited by 312
GANalyze: Toward Visual Definitions of Cognitive Image Properties
Goetschalckx, L., Andonian, A., Oliva, A., & Isola, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00584 · Cited by 283
Self-citationConcept whitening for interpretable image recognition
Chen, Z., Bei, Y., & Rudin, C. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00265-z · Cited by 279
Deep Learning: The Good, the Bad, and the Ugly
Serre, T. (2019) · Annual Review of Vision Science · doi:10.1146/annurev-vision-091718-014951 · Cited by 267
Interpretable Basis Decomposition for Visual Explanation
Zhou, B., Sun, Y., Bau, D., & Torralba, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_8 · Cited by 260
Self-citationExplaining the black-box model: A survey of local interpretation methods for deep neural networks
Liang, Y., Li, S., Yan, C., Li, M., & Jiang, C. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.08.011 · Cited by 253
Deep neural networks in computational neuroscience
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019) · Oxford Research Encyclopedia of Neuroscience · doi:10.1093/acrefore/9780190264086.013.46 · Cited by 186
Visual Imagery and Perception Share Neural Representations in the Alpha Frequency Band
Xie, S., Kaiser, D., & Cichy, R. M. (2020) · Current Biology · doi:10.1016/j.cub.2020.04.074 · Cited by 139
Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
Chu, L., Hu, X., Hu, J., Wang, L., & Pei, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220063 · Cited by 87
Nam, W.-J., Gur, S., Choi, J., Wolf, L., & Lee, S.-W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5632 · Cited by 98
Mohamed, E., Sirlantzis, K., & Howells, G. (2022) · Displays · doi:10.1016/j.displa.2022.102239 · Cited by 67
Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization
Gur, S., Ali, A., & Wolf, L. (2020) · arXiv · doi:10.48550/arxiv.2012.02166 · Cited by 34
“…classes include those that generate salient feature maps (Dabkowski and Gal 2017; Simonyan, Vedaldi, and Zisserman 2013; Mahendran and Vedaldi 2016; Zhou et al. 2016; Zeiler and Fergus 2014; Zhou et al. 2018), Activation Maximization (Erhan et al. 2009) and Excitation Backprop (Zhang et al. 2018).”
BackgroundEmergence of Visual Center-Periphery Spatial Organization in Deep Convolutional Neural Networks
Mohsenzadeh, Y., Mullin, C., Lahner, B., & Oliva, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-61409-0 · Cited by 31
Self-citationAre there any ‘object detectors’ in the hidden layers of CNNs trained to identify objects or scenes?
Gale, E. M., Martin, N., Blything, R., Nguyen, A., & Bowers, J. S. (2020) · Vision Research · doi:10.1016/j.visres.2020.06.007 · Cited by 16
Metadata
- Authors
- Zhou, B., Bau, D., Oliva, A., & Torralba, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-09-01
- Volume
- 41
- Issue
- 9
- Pages
- 2131-2145
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2963081790
- Cited by
- 330
- References
- 54
- FWCI
- 19.52
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.