The DOI matches its DataCite record; 7 of the 7 references it cites were found in a registry or an index.
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Open accessAbstract
This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets). We consider two visualisation techniques, based on computing the gradient of the class score with respect to the input image. The first one generates an image, which maximises the class score [Erhan et al., 2009], thus visualising the notion of the class, captured by a ConvNet. The second technique computes a class saliency map, specific to a given image and class. We show that such maps can be employed for weakly supervised object segmentation using classification ConvNets. Finally, we establish the connection between the gradient-based ConvNet visualisation methods and deconvolutional networks [Zeiler et al., 2013].
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“saliency maps (Simonyan et al., 2013), simple representations reporting the gradient of a target class score with respect to each input pixel, have been recently transposed to P300 decoding (Farahat et al., 2019).”
From A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision (2021) · cites this paper as Simonyan et al., 2013
In the citing paper
“Barring a few exceptions (Caruana et al., 2015; Lundberg et al., 2018b; Ustun et al., 2019; Jesus et al., 2021), much of the existing work is designed and developed for benchmark classification problems, often with synthetic data and usually validated with user studies limited to users in research settings such as Amazon Mechanical Turk (AMT; Simonyan et al., 2013; Zeiler and Fergus, 2014; Bach et”
From Explainable machine learning for public policy: Use cases, gaps, and research directions (2023) · cites this paper as Simonyan et al., 2013
In the citing paper
“Borzoi enables direct characterization of tissue-specific cis-regulatory TF motifs by applying attribution methods to the predicted RNA-seq coverage statistics45–50.”
From Predicting RNA-seq coverage from DNA sequence as a unifying model of gene regulation (2025) · cites this paper as [46]
In the citing paper
“Another common approach that is used to shed light on the predictions of Deep CNN is based on creating saliency maps [132] and guided backpropagation [133], [134].”
From AI in Medical Imaging Informatics: Current Challenges and Future Directions (2020) · cites this paper as [132]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images
Boykov, Y. Y., & Jolly, M.-P. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937505 · Cited by 3,674
A discriminatively trained, multiscale, deformable part model
Felzenszwalb, P. F., McAllester, D., & Ramanan, D. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587597 · Cited by 2,972
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 646
Visualizing and Understanding Convolutional Networks
Zeiler, M. D., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1311.2901 · Cited by 522
Research works printed but not linked: 4
- [1][1] D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R. M¨ uller. How to explain individual classification decisions. JMLR , 11:1803-1831, 2010. -
- [4][4] D. C. Ciresan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. In Proc. CVPR , pages 3642-3649, 2012. -
- [8][8] A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS , pages 1106-1114, 2012. -
- [12][12] K. Simonyan, A. Vedaldi, and A. Zisserman. Deep Fisher networks and class saliency maps for object classification and localisation. In ILSVRC workshop , 2013. URL http://image-net.org/ challenges/LSVRC/2013/slides/ILSVRC\_az.pdf . -
Cited by
Sorted by citations
Items 1,001–1,056 of 1,056
Zanisi, L., Huertas-Company, M., Lanusse, F., Bottrell, C., Pillepich, A., Nelson, D., Rodriguez-Gomez, V., Shankar, F., Hernquist, L., Dekel, A., et al. (2021) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/staa3864 · Cited by 65
Lin, Y.-S., Lee, W.-C., & Celik, Z. B. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467213 · Cited by 64
Towards relatable explainable AI with the perceptual process
Zhang, W., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501826 · Cited by 67
Li, J., Wang, J., Zhang, P., Wang, R., Mei, Y., Sun, Z., Fei, L., Jiang, M., Ma, L., E, W., et al. (2022) · Nature Genetics · doi:10.1038/s41588-022-01197-7 · Cited by 57
Predicting phenotypes from genetic, environment, management, and historical data using CNNs
Washburn, J. D., Cimen, E., Ramstein, G., Reeves, T., O’Briant, P., McLean, G., Cooper, M., Hammer, G., & Buckler, E. S. (2021) · Theoretical and Applied Genetics · doi:10.1007/s00122-021-03943-7 · Cited by 57
Security and Privacy Issues in Deep Learning
Bae, H., Jang, J., Jung, D., Jang, H., Ha, H., Lee, H., & Yoon, S. (2018) · arXiv · doi:10.48550/arxiv.1807.11655 · Cited by 56
Explaining Neural Networks Semantically and Quantitatively
Chen, R., Chen, H., Huang, G., Ren, J., & Zhang, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00928 · Cited by 56
The need to move away from agential-AI: Empirical investigations, useful concepts and open issues
Cabitza, F., Campagner, A., & Simone, C. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102696 · Cited by 55
Kick, D. R., Wallace, J. G., Schnable, J. C., Kolkman, J. M., Alaca, B., Beissinger, T. M., Edwards, J., Ertl, D., Flint-Garcia, S., Gage, J. L., et al. (2023) · G3: Genes|Genomes|Genetics · doi:10.1093/g3journal/jkad006 · Cited by 53
Visual Scene Understanding for Autonomous Driving Using Semantic Segmentation
Hofmarcher, M., Unterthiner, T., Arjona-Medina, J., Klambauer, G., Hochreiter, S., & Nessler, B. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_15 · Cited by 51
Mitigating the Backdoor Attack by Federated Filters for Industrial IoT Applications
Hou, B., Gao, J., Guo, X., Baker, T., Zhang, Y., Wen, Y., & Liu, Z. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3112100 · Cited by 49
Event-based media processing and analysis: A survey of the literature
Tzelepis, C., Ma, Z., Mezaris, V., Ionescu, B., Kompatsiaris, I., Boato, G., Sebe, N., & Yan, S. (2016) · Image and Vision Computing · doi:10.1016/j.imavis.2016.05.005 · Cited by 49
A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks
Grün, F., Rupprecht, C., Navab, N., & Tombari, F. (2016) · arXiv · doi:10.48550/arxiv.1606.07757 · Cited by 49
Mitigating Gender Bias in Captioning Systems
Tang, R., Du, M., Li, Y., Liu, Z., Zou, N., & Hu, X. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449950 · Cited by 50
Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discovery
Wagner, P., Mehari, T., Haverkamp, W., & Strodthoff, N. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108525 · Cited by 48
Abduction-Based Explanations for Machine Learning Models
Ignatiev, A., Narodytska, N., & Marques-Silva, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011511 · Cited by 40
Borra, D., Fantozzi, S., & Magosso, E. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.655840 · Cited by 36
Kabir, S., Hossain, M. S., & Andersson, K. (2025) · Algorithms · doi:10.3390/a18090556 · Cited by 44
Do Not Trust Additive Explanations
Gosiewska, A., & Biecek, P. (2019) · arXiv · doi:10.48550/arxiv.1903.11420 · Cited by 34
Explainable outlier detection: What, for Whom and Why?
Sejr, J. H., & Schneider-Kamp, A. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100172 · Cited by 32
PerturbNet predicts single-cell responses to unseen chemical and genetic perturbations
Yu, H., Qian, W., Song, Y., & Welch, J. D. (2025) · Molecular Systems Biology · doi:10.1038/s44320-025-00131-3 · Cited by 57
Extendable and explainable deep learning for pan-cancer radiogenomics research
Liu, Q., & Hu, P. (2022) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.102111 · Cited by 27
Kumar, D., Sankar, V., Clausi, D., Taylor, G. W., & Wong, A. (2019) · IEEE Access · doi:10.1109/access.2019.2945524 · Cited by 26
Mirzaei, S., Mao, H., Al-Nima, R. R. O., & Woo, W. L. (2023) · Information · doi:10.3390/info15010004 · Cited by 25
Robustness Analysis of Deep Learning-Based Lung Cancer Classification Using Explainable Methods
Malafaia, M., Silva, F., Neves, I., Pereira, T., & Oliveira, H. P. (2022) · IEEE Access · doi:10.1109/access.2022.3214824 · Cited by 25
Coupland, H., Scheidwasser, N., Katsiferis, A., Davies, M., Flaxman, S., Hulvej Rod, N., Mishra, S., Bhatt, S., & Unwin, H. J. T. (2025) · BMC Public Health · doi:10.1186/s12889-025-22705-4 · Cited by 26
A survey of post-hoc xai methods from a visualization perspective: Challenges and opportunities
Bhati, D., Amiruzzaman, M., Zhao, Y., Guercio, A., & Le, T. (2025) · IEEE Access · doi:10.1109/access.2025.3581136 · Cited by 24
MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset
Meng, C., Trinh, L., Xu, N., & Liu, Y. (2021) · doi:10.21203/rs.3.rs-402058/v1 · Cited by 21
Blind Users Accessing Their Training Images in Teachable Object Recognizers
Hong, J., Gandhi, J., Mensah, E. E., Zeraati, F. Z., Jarjue, E., Lee, K., & Kacorri, H. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544824 · Cited by 22
Angrick, M., Luo, S., Rabbani, Q., Candrea, D. N., Shah, S., Milsap, G. W., Anderson, W. S., Gordon, C. R., Rosenblatt, K. R., Clawson, L., et al. (2023) · doi:10.1101/2023.06.30.23291352 · Cited by 95
On Utility and Privacy in Synthetic Genomic Data
Oprisanu, B., Ganev, G., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24092 · Cited by 23
Binder, A., Weber, L., Lapuschkin, S., Montavon, G., Müller, K.-R., & Samek, W. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01549 · Cited by 21
Graph convolutional networks for epigenetic state prediction using both sequence and 3D genome data
Lanchantin, J., & Qi, Y. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa793 · Cited by 17
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations
Salewski, L., Koepke, A. S., Lensch, H. P. A., & Akata, Z. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_5 · Cited by 20
Weakly Supervised Learning of Placental Ultrasound Images with Residual Networks
Qi, H., Collins, S., & Noble, A. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-60964-5_9 · Cited by 17
Deep learning to quantify the pace of brain aging in relation to neurocognitive changes
Yin, C., Imms, P., Chowdhury, N. F., Chaudhari, N. N., Ping, H., Wang, H., Bogdan, P., Irimia, A., Initiative, T. A. D. N., Weiner, M., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2413442122 · Cited by 18
ContextCite: Attributing Model Generation to Context
Cohen-Wang, B., Shah, H., Georgiev, K., & Madry, A. (2024) · arXiv · doi:10.48550/arxiv.2409.00729 · Cited by 16
“For example, if a system like ChatGPT browses the web to answer a question about the news, it may end up retrieving a poisoned article and adding it to the language model’s context.”
LS-Tree: Model Interpretation When the Data Are Linguistic
Chen, J., & Jordan, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5749 · Cited by 14
Weakly- and Semi-supervised Evidence Extraction
Pruthi, D., Dhingra, B., Neubig, G., & Lipton, Z. C. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.353 · Cited by 13
Rationales for Sequential Predictions
Vafa, K., Deng, Y., Blei, D., & Rush, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.807 · Cited by 14
Gauge fixing for sequence-function relationships
Posfai, A., Zhou, J., McCandlish, D. M., & Kinney, J. B. (2024) · doi:10.1101/2024.05.12.593772 · Cited by 11
Yuan, H., Hong, C., Jiang, P.-T., Zhao, G., Tran, N. T. A., Xu, X., Yan, Y. Y., & Liu, N. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104673 · Cited by 11
Sources of Opacity in Computer Systems: Towards a Comprehensive Taxonomy
Mann, S., Crook, B., Kästner, L., Schomäcker, A., & Speith, T. (2023) · 2023 IEEE 31st International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00063 · Cited by 10
Arends, B. K. O., van Amsterdam, W. A. C., van der Harst, P., van Smeden, M., van Es, R., & van de Leur, R. R. (2026) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztag038 · Cited by 6
Assessing the State of Self-Supervised Human Activity Recognition using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · arXiv · Cited by 107
SetQuence & SetOmic: Deep Set Transformer-based Representations of Cancer Multi-Omics
Jurenaite, N., León-Periñán, D., Donath, V., Torge, S., & Jäkel, R. (2022) · 2022 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb55180.2022.9863058 · Cited by 5
Sanity Checks for Saliency Metrics
Tomsett, R., Harborne, D., Chakraborty, S., Gurram, P., & Preece, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6064 · Cited by 3
Rezaeian, O., Bayrak, A. E., & Asan, O. (2025) · arXiv · doi:10.48550/arxiv.2501.16693 · Cited by 39
Nomological Deductive Reasoning for Trustworthy, Human-Readable, and Actionable AI Outputs
Hakizimana, G., & Ledezma Espino, A. (2025) · Algorithms · doi:10.3390/a18060306 · Cited by 2
Toward Scalable and Unified Example-Based Explanation and Outlier Detection
Chong, P., Cheung, N.-M., Elovici, Y., & Binder, A. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3127847 · Cited by 14
EMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledge
Palash, M., & Bhargava, B. (2023) · arXiv · doi:10.48550/arxiv.2306.08657 · Cited by 25
“Additionally, in [16], the authors employed Saliency Map and Grad-CAM methods to generate an explainable model for driver’s facial emotion recognition.”
MethodPerturbNet predicts single-cell responses to unseen chemical and genetic perturbations
Yu, H., & Welch, J. D. (2022) · doi:10.1101/2022.07.20.500854 · Cited by 57
“841 Previous attempts to interpret neural network models have focused on gradients [57, 58] and back-propagation [59, 60].”
BackgroundJetley, S., Lord, N. A., Lee, N., & Torr, P. H. S. (2018) · arXiv · doi:10.48550/arxiv.1804.02391 · Cited by 383
Explainable Deep One-Class Classification
Liznerski, P., Ruff, L., Vandermeulen, R. A., Franks, B. J., Kloft, M., & Müller, K.-R. (2020) · arXiv · doi:10.48550/arxiv.2007.01760 · Cited by 185
(2020) · neural information processing systems · Cited by 100
Debugging Tests for Model Explanations
Adebayo, J., Muelly, M., Liccardi, I., & Kim, B. (2020) · arXiv · doi:10.48550/arxiv.2011.05429 · Cited by 88
Metadata
- Authors
- Simonyan, K., Vedaldi, A., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2013
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1312.6034
- OpenAlex
- W2169393322
- Cited by
- 5,370
- References
- 7
- Citations by year
- 2026: 53 · 2025: 259 · 2024: 513 · 2023: 605 · 2022: 584 · 2021: 1000 · 2020: 776 · 2019: 650 · 2018: 434 · 2017: 250 · 2016: 150 · 2015: 73 · 2014: 19
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- image classification, saliency maps, class activation maps, deconvolutional networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
