The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Towards Deeper Graph Neural Networks
Open accessAbstract
Graph neural networks have shown significant success in the field of graph representation learning. Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Nevertheless, one layer of these neighborhood aggregation methods only consider immediate neighbors, and the performance decreases when going deeper to enable larger receptive fields. Several recent studies attribute this performance deterioration to the over-smoothing issue, which states that repeated propagation makes node representations of different classes indistinguishable. In this work, we study this observation systematically and develop new insights towards deeper graph neural networks. First, we provide a systematical analysis on this issue and argue that the key factor compromising the performance significantly is the entanglement of representation transformation and propagation in current graph convolution operations. After decoupling these two operations, deeper graph neural networks can be used to learn graph node representations from larger receptive fields. We further provide a theoretical analysis of the above observation when building very deep models, which can serve as a rigorous and gentle description of the over-smoothing issue. Based on our theoretical and empirical analysis, we propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields. A set of experiments on citation, co-authorship, and co-purchase datasets have confirmed our analysis and insights and demonstrated the superiority of our proposed methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
(2010) · International Conference on Machine Learning · Cited by 13,995
(1999) · Cited by 12,854
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
(2017) · Cited by 11,625
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Neural Message Passing for Quantum Chemistry
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., & Dahl, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01212 · Cited by 3,440
Collective Classification in Network Data
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., & Eliassi‐Rad, T. (2008) · AI Magazine · doi:10.1609/aimag.v29i3.2157 · Cited by 3,293
Formal Semantics for Kolmogorov-Arnold Network Representations of Operational Games
Zhukov, G. A. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.15206559 · Cited by 3,212
Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning
Li, Q., Han, Z., & Wu, X.-M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11604 · Cited by 2,872
Image-Based Recommendations on Styles and Substitutes
McAuley, J., Targett, C., Shi, Q., & van den Hengel, A. (2015) · Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2766462.2767755 · Cited by 2,280
A signal processing approach to fair surface design
Taubin, G. (1995) · Proceedings of the 22nd annual conference on Computer graphics and interactive techniques - SIGGRAPH '95 · doi:10.1145/218380.218473 · Cited by 1,855
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs
Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., & Bronstein, M. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.576 · Cited by 1,820
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Defferrard, M., Bresson, X., & Vandergheynst, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.09375 · Cited by 1,673
An End-to-End Deep Learning Architecture for Graph Classification
Zhang, M., Cui, Z., Neumann, M., & Chen, Y. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11782 · Cited by 1,461
Random Walks on Graphs: A Survey
Lovász, L. (2001) · Cited by 1,411
Simplifying Graph Convolutional Networks
Wu, F., Zhang, T., Junior, A. H. D. S., Fifty, C., Yu, T., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07153 · Cited by 1,165
Fast Graph Representation Learning with PyTorch Geometric
Fey, M., & Lenssen, J. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02428 · Cited by 1,132
Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]
Chapelle, O., Scholkopf, B., & Zien, E. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2015974 · Cited by 1,066
Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., & Sun, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5747 · Cited by 1,043
Stochastic systems : estimation, identification, and adaptive control
Kumar, P. R., & Varaiya, P. (1986) · Medical Entomology and Zoology · Cited by 1,018
Representation Learning on Graphs with Jumping Knowledge Networks
Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K., & Jegelka, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03536 · Cited by 891
Hierarchical Graph Representation Learning with Differentiable Pooling
Ying, R., You, J., Morris, C., Ren, X., Hamilton, W. L., & Leskovec, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08804 · Cited by 798
Predict then Propagate: Graph Neural Networks meet Personalized PageRankPreprint
Klicpera, J., Bojchevski, A., & Günnemann, S. (2018) · arXiv (Cornell University) · Cited by 756
Large-Scale Learnable Graph Convolutional Networks
Gao, H., Wang, Z., & Ji, S. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219947 · Cited by 638
Self-citationShchur, O., Mumme, M., Bojchevski, A., & Günnemann, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05868 · Cited by 588
Non-Negative Matrices and Markov Chains.
Isham, V., & Seneta, E. (1983) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2982031 · Cited by 439
Geom-GCN: Geometric Graph Convolutional Networks
Pei, H., Wei, B., Chang, K. C., Yu, L., & Yang, B. X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05287 · Cited by 396
Graph Convolutional Networks with EigenPooling
Ma, Y., Wang, S., Aggarwal, C. C., & Tang, J. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330982 · Cited by 320
Link Prediction Based on Graph Neural Networks
Zhang, M., & Chen, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09691 · Cited by 263
Graph U-NetsPreprint
Gao, H., & Ji, S. (2019) · arXiv (Cornell University) · Cited by 231
Self-citationWeisfeiler-Lehman Neural Machine for Link Prediction
Zhang, M., & Chen, Y. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3097996 · Cited by 216
StructPool: Structured Graph Pooling via Conditional Random Fields
Yuan, H., & Ji, S. (2020) · International Conference on Learning Representations · Cited by 146
Self-citationA Multi-Scale Approach for Graph Link Prediction
Cai, L., & Ji, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5731 · Cited by 132
Self-citation
Cited by
Graph Neural Networks in Recommender Systems: A Survey
Wu, S., Sun, F., Zhang, W., Xie, X., & Cui, B. (2023) · ACM Computing Surveys · doi:10.1145/3535101 · Cited by 1,326
Graph Contrastive Learning with Augmentations
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., & Shen, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13902 · Cited by 824
Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
Shi, Y., Huang, Z., Feng, S., Zhong, H., Wang, W., & Sun, Y. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/214 · Cited by 751
Explainability in Graph Neural Networks: A Taxonomic Survey
Yuan, H., Yu, H., Gui, S., & Ji, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204236 · Cited by 583
Self-citationSelf-Supervised Learning of Graph Neural Networks: A Unified Review
Xie, Y., Xu, Z., Zhang, J., Wang, Z., & Ji, S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3170559 · Cited by 389
Self-citation
Metadata
- Authors
- Liu, M., Gao, H., & Ji, S.
- Venue
- Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2020-08-20
- Pages
- 338-348
- OpenAlex
- W3042770487
- Cited by
- 507
- References
- 37
- FWCI
- 27.62
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.