CoChat
Sign Up
PreprintCited by 850Explainable Artificial Intelligence (XAI)Advanced Graph Neural NetworksPrivacy-Preserving Technologies in Data

GNNExplainer: Generating Explanations for Graph Neural Networks

Open access
Ying, Rex, Bourgeois, Dylan, You, Jiaxuan, Zitnik, Marinka, Leskovec, Jure·2019·arXiv
DOI 10.48550/arxiv.1903.03894arXiv 1903.03894
Open this paper in CoChat — free accountOpen PDF

Abstract

Graph Neural Networks (GNNs) are a powerful tool for machine learning on graphs.GNNs combine node feature information with the graph structure by recursively passing neural messages along edges of the input graph. However, incorporating both graph structure and feature information leads to complex models, and explaining predictions made by GNNs remains unsolved. Here we propose GNNExplainer, the first general, model-agnostic approach for providing interpretable explanations for predictions of any GNN-based model on any graph-based machine learning task. Given an instance, GNNExplainer identifies a compact subgraph structure and a small subset of node features that have a crucial role in GNN's prediction. Further, GNNExplainer can generate consistent and concise explanations for an entire class of instances. We formulate GNNExplainer as an optimization task that maximizes the mutual information between a GNN's prediction and distribution of possible subgraph structures. Experiments on synthetic and real-world graphs show that our approach can identify important graph structures as well as node features, and outperforms baselines by 17.1% on average. GNNExplainer provides a variety of benefits, from the ability to visualize semantically relevant structures to interpretability, to giving insights into errors of faulty GNNs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1903.03894

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 39

  1. [2][2] J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim. Sanity checks for saliency maps. In NeurIPS , 2018. -
  2. [3][3] M. Gethsiyal Augasta and T. Kathirvalavakumar. Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems. Neural Processing Letters , 35(2):131-150, April 2012. -
  3. [4][4] Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261 , 2018. - · arXiv
  4. [5][5] J. Chen, J. Zhu, and L. Song. Stochastic training of graph convolutional networks with variance reduction. In ICML , 2018. -
  5. [6][6] Jianbo Chen, Le Song, Martin J Wainwright, and Michael I Jordan. Learning to explain: An information-theoretic perspective on model interpretation. arXiv preprint arXiv:1802.07814 , 2018. - · arXiv
  6. [7][7] Jie Chen, Tengfei Ma, and Cao Xiao. Fastgcn: fast learning with graph convolutional networks via importance sampling. In ICLR , 2018. -
  7. [8][8] Z. Chen, L. Li, and J. Bruna. Supervised community detection with line graph neural networks. In ICLR , 2019. -
  8. [9][9] E. Cho, S. Myers, and J. Leskovec. Friendship and mobility: user movement in location-based social networks. In KDD , 2011. -
  9. [10][10] A. Debnath et al. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry , 34(2):786-797, 1991. -
  10. [12][12] D. Duvenaud et al. Convolutional networks on graphs for learning molecular fingerprints. In NIPS , 2015. -
  11. [13][13] D. Erhan, Y. Bengio, A. Courville, and P. Vincent. Visualizing higher-layer features of a deep network. University of Montreal , 1341(3):1, 2009. -
  12. [14][14] A. Fisher, C. Rudin, and F. Dominici. All Models are Wrong but many are Useful: Variable Importance for Black-Box, Proprietary, or Misspecified Prediction Models, using Model Class Reliance. January 2018. arXiv: 1801.01489. - · arXiv
  13. [15][15] R. Guidotti et al. A Survey of Methods for Explaining Black Box Models. ACM Comput. Surv. , 51(5):93:1-93:42, 2018. -
  14. [16][16] W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. In NIPS , 2017. -
  15. [17][17] G. Hooker. Discovering additive structure in black box functions. In KDD , 2004. -
  16. [18][18] W.B. Huang, T. Zhang, Y. Rong, and J. Huang. Adaptive sampling towards fast graph representation learning. In NeurIPS , 2018. -
  17. [19][19] Bo Kang, Jefrey Lijffijt, and Tijl De Bie. Explaine: An approach for explaining network embedding-based link predictions. arXiv:1904.12694 , 2019. - · arXiv
  18. [20][20] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In NeurIPS , 2013. -
  19. [21][21] T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR , 2016. -
  20. [22][22] Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In ICML , 2018. -
  21. [23][23] P. W. Koh and P. Liang. Understanding black-box predictions via influence functions. In ICML , 2017. -
  22. [24][24] Srijan Kumar, William L Hamilton, Jure Leskovec, and Dan Jurafsky. Community interaction and conflict on the web. In WWW , pages 933-943, 2018. -
  23. [26][26] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. Gated graph sequence neural networks. arXiv:1511.05493 , 2015. - · arXiv
  24. [28][28] D. Neil et al. Interpretable Graph Convolutional Neural Networks for Inference on Noisy Knowledge Graphs. In ML4H Workshop at NeurIPS , 2018. -
  25. [29][29] M. Ribeiro, S. Singh, and C. Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In KDD , 2016. -
  26. [30][30] G. J. Schmitz, C. Aldrich, and F. S. Gouws. ANN-DT: an algorithm for extraction of decision trees from artificial neural networks. IEEE Transactions on Neural Networks , 1999. -
  27. [31][31] A. Shrikumar, P. Greenside, and A. Kundaje. Learning Important Features Through Propagating Activation Differences. In ICML , 2017. -
  28. [32][32] M. Sundararajan, A. Taly, and Q. Yan. Axiomatic Attribution for Deep Networks. In ICML , 2017. -
  29. [36][36] K. Xu, C. Li, Y. Tian, T. Sonobe, K. Kawarabayashi, and S. Jegelka. Representation learning on graphs with jumping knowledge networks. In ICML , 2018. -
  30. [37][37] Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In KDD , pages 1365-1374. ACM, 2015. -
  31. [38][38] C. Yeh, J. Kim, I. Yen, and P. Ravikumar. Representer point selection for explaining deep neural networks. In NeurIPS , 2018. -
  32. [39][39] R. Ying, R. He, K. Chen, P. Eksombatchai, W. Hamilton, and J. Leskovec. Graph convolutional neural networks for web-scale recommender systems. In KDD , 2018. -
  33. [40][40] Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec. Hierarchical graph representation learning with differentiable pooling. In NeurIPS , 2018. -
  34. [42][42] J. You, Rex Ying, and J. Leskovec. Position-aware graph neural networks. In ICML , 2019. -
  35. [43][43] M. Zeiler and R. Fergus. Visualizing and Understanding Convolutional Networks. In ECCV . 2014. -
  36. [44][44] M. Zhang and Y. Chen. Link prediction based on graph neural networks. In NIPS , 2018. -
  37. [45][45] Z. Zhang, Peng C., and W. Zhu. Deep Learning on Graphs: A Survey. arXiv:1812.04202 , 2018. - · arXiv
  38. [47][47] J. Zilke, E. Loza Mencia, and F. Janssen. DeepRED - Rule Extraction from Deep Neural Networks. In Discovery Science . Springer International Publishing, 2016. -
  39. [48][48] L. Zintgraf, T. Cohen, T. Adel, and M. Welling. Visualizing deep neural network decisions: Prediction difference analysis. In ICLR , 2017. -

Cited by

Sorted by citations

Authors
Ying, R., Bourgeois, D., You, J., Zitnik, M., & Leskovec, J.
Affiliations
Stanford University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1903.03894
arXiv
1903.03894
OpenAlex
W2972317931
Cited by
850
References
7
Citations by year
2026: 19 · 2025: 161 · 2024: 240 · 2023: 179 · 2022: 126 · 2021: 99 · 2020: 21 · 2019: 5
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
graph neural networks, explainable AI, model-agnostic explanation, subgraph extraction, mutual information maximization, interpretability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review