Improving Graph Attention Networks with Large Margin-based Constraints
Open accessAbstract
Graph Attention Networks (GATs) are the state-of-the-art neural architecture for representation learning with graphs. GATs learn attention functions that assign weights to nodes so that different nodes have different influences in the feature aggregation steps. In practice, however, induced attention functions are prone to over-fitting due to the increasing number of parameters and the lack of direct supervision on attention weights. GATs also suffer from over-smoothing at the decision boundary of nodes. Here we propose a framework to address their weaknesses via margin-based constraints on attention during training. We first theoretically demonstrate the over-smoothing behavior of GATs and then develop an approach using constraint on the attention weights according to the class boundary and feature aggregation pattern. Furthermore, to alleviate the over-fitting problem, we propose additional constraints on the graph structure. Extensive experiments and ablation studies on common benchmark datasets demonstrate the effectiveness of our method, which leads to significant improvements over the previous state-of-the-art graph attention methods on all datasets.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This argument has been pointed out in several papers, where it is proposed to improve conventional graph attention mechanisms by projecting the concatenated features once onto the learning matrix.
“This argument has already been pointed out in several papers, e.g., in [91][92][93], where it is proposed to improve the performance of conventional graph attention mechanisms by projecting the concatenated features once onto the learning matrix.”
From TVGeAN: Tensor Visibility Graph-Enhanced Attention Network for Versatile Multivariant Time Series Learning Tasks (2024) · cites this paper as [92]
The Constrained Graph Attention Networks section (p. 4) proposes improving GAT through margin-based constraints and selective neighbor aggregation. The Analysis of GATs section (p. 2) describes concatenating transformed node features in the conventional attention function, but does not propose projecting concatenated features once onto a learning matrix as the improvement. (LLM, conf=0.88)
Experimental — accuracy not yet measuredSeveral GAT variants have been developed to overcome GAT limitations using strategies such as multi-headed attention, restricting aggregation to top-K neighbors, and reordering GAT operations.
“Some notable examples are the use of multi-headed attention mechanisms at each node of the GNN [91], the restriction of aggregation to the neighbors with the highest K-attention scores [92], and the reordering of GAT operations [93].”
From TVGeAN: Tensor Visibility Graph-Enhanced Attention Network for Versatile Multivariant Time Series Learning Tasks (2024) · cites this paper as [92]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · see citing works
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203 · see citing works
Graph Attention NetworksPreprint
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 12,099 · see citing works
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639 · see citing works
Self-citationAdvances In Deep Learning On Graphs (Gsp'18 Workshop)Preprint
Defferrard, M., Bresson, X., & Vandergheynst, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.09375 · Cited by 6,912 · see citing works
Neural Message Passing for Quantum ChemistryPreprint
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., & Dahl, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01212 · Cited by 5,414 · see citing works
Spectral Networks and Locally Connected Networks on Graphs
Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6203 · Cited by 3,983 · see citing works
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219890 · Cited by 3,733 · see citing works
Self-citationA Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. (2020) · UTS ePRESS (University of Technology Sydney) · Cited by 3,174 · see citing works
Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning
Li, Q., Han, Z., & Wu, X.-M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11604 · Cited by 2,872 · see citing works
Convolutional Networks on Graphs for Learning Molecular Fingerprints
Duvenaud, D., Maclaurin, D., Aguilera-Iparraguirre, J., Gómez-Bombarelli, R., Hirzel, T., Aspuru-Guzik, A., & Adams, R. P. (2015) · arXiv · doi:10.48550/arxiv.1509.09292 · Cited by 2,628 · see citing works
Learning Convolutional Neural Networks for GraphsPreprint
Niepert, M., Ahmed, M., & Kutzkov, K. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1605.05273 · Cited by 2,311 · see citing works
Simplifying Graph Convolutional NetworksPreprint
Wu, F., Zhang, T., Junior, A. H. D. S., Fifty, C., Yu, T., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07153 · Cited by 2,251 · see citing works
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications
Cai, H., Zheng, V. W., & Chang, K. C.-C. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2807452 · Cited by 2,034 · see citing works
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs
Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., & Bronstein, M. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.576 · Cited by 1,820 · see citing works
Link Prediction Based on Graph Neural NetworksPreprint
Zhang, M., & Chen, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09691 · Cited by 1,394 · see citing works
Deep Convolutional Networks on Graph-Structured Data
Henaff, M., Bruna, J., & LeCun, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.05163 · Cited by 1,255 · see citing works
Graph Convolutional Matrix CompletionPreprint
van den Berg, R., Kipf, T., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02263 · Cited by 1,023 · see citing works
Interaction Networks for Learning about Objects, Relations and Physics
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1612.00222 · Cited by 969 · see citing works
Large-Scale Learnable Graph Convolutional Networks
Gao, H., Wang, Z., & Ji, S. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219947 · Cited by 638 · see citing works
GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal GraphsPreprint
Zhang, J., Shi, X., Xie, J., Ma, H., King, I., & Yeung, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07294 · Cited by 580 · see citing works
Deep Learning via Semi-supervised Embedding
Weston, J., Ratle, F., Mobahi, H., & Collobert, R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_34 · Cited by 577 · see citing works
Diffusion-Convolutional Neural Networks
Atwood, J. R., & Towsley, D. (2016) · neural information processing systems · Cited by 566 · see citing works
Laplacians and the Cheeger Inequality for Directed Graphs
Chung, F. (2005) · Annals of Combinatorics · doi:10.1007/s00026-005-0237-z · Cited by 535 · see citing works
Attention-based Graph Neural Network for Semi-supervised LearningPreprint
Thekumparampil, K. K., Wang, C., Oh, S., & Li, L.-J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03735 · Cited by 365 · see citing works
Adaptive Sampling Towards Fast Graph Representation LearningPreprint
Huang, W., Zhang, T., Rong, Y., & Huang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05343 · Cited by 365 · see citing works
Self-citationGeniePath: Graph Neural Networks with Adaptive Receptive Paths
Liu, Z., Chen, C., Li, L., Zhou, J., Li, X., Song, L., & Qi, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014424 · Cited by 350 · see citing works
Graph Classification using Structural Attention
Lee, J. B., Rossi, R. A., & Kong, X. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3219980 · Cited by 288 · see citing works
Stochastic Training of Graph Convolutional Networks with Variance ReductionPreprint
Chen, J., Zhu, J., & Song, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10568 · Cited by 283 · see citing works
Watch Your Step: Learning Node Embeddings via Graph AttentionPreprint
Abu-El-Haija, S., Perozzi, B., Al‐Rfou, R., & Alemi, A. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.09599 · Cited by 181 · see citing works
Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
Tu, M., Wang, G., Huang, J., Tang, Y., He, X., & Zhou, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1260 · Cited by 166 · see citing works
Self-citationLearning Steady-States of Iterative Algorithms over Graphs
Dai, H., Kozareva, Z., Dai, B., Smola, A. J., & Song, L. (2018) · International Conference on Machine Learning · Cited by 151 · see citing works
Ryu, S., Lim, J., Hong, S. H., & Kim, W. Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10988 · Cited by 96 · see citing works
Rapidly mixing Markov chains with applications in computer science and physics
Randall, D. (2006) · Computing in Science & Engineering · doi:10.1109/mcse.2006.30 · Cited by 92 · see citing works
Question answering by reasoning across documents with graph convolutional networks
De Cao, N., Aziz, W., & Titov, I. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1240 · Cited by 54 · see citing works
Graph Neural Networks and Boolean SatisfiabilityPreprint
Bünz, B., & Lamm, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03592 · Cited by 17 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [Defferrard, Bresson, and Vandergheynst 2016] Defferrard, M.; Bresson, X.; and Vandergheynst, P. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS , 3844-3852.
- [Lov´ asz and others 1993] Lov´ asz, L., et al. 1993. Random walks on graphs: A survey. Combinatorics, Paul erdos is eighty 2(1):1-46.
- [Sen et al. 2008] Sen, P.; Namata, G.; Bilgic, M.; Getoor, L.; Galligher, B.; and Eliassi-Rad, T. 2008. Collective classification in network data. AI magazine 29(3):93-106.
- [Zitnik and Leskovec 2017] Zitnik, M., and Leskovec, J. 2017. Predicting multicellular function through multi-layer tissue networks. Bioinformatics 33(14):i190-i198.
Cited by
Sorted by citations
Principal Neighbourhood Aggregation for Graph NetsPreprint
Corso, G., Cavalleri, L., Beaini, D., Lió, P., & Veličković, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05718 · Cited by 308 · see citing works
Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
Yan, Y., Hashemi, M., Swersky, K., Yang, Y. L., & Koutra, D. (2022) · Proceedings · doi:10.1109/icdm54844.2022.00169 · Cited by 205 · see citing works
How to Find Your Friendly Neighborhood: Graph Attention Design with Self-SupervisionPreprint
Kim, D., & Oh, A. H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.04879 · Cited by 181 · see citing works
Multi-hop Attention Graph Neural Networks
Wang, G., Ying, R., Huang, J., & Leskovec, J. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/425 · Cited by 164 · see citing works
Self-citationBootstrapped Representation Learning on GraphsPreprint
Thakoor, S., Tallec, C., Azar, M. G., Munos, R., Veličković, P., & Valko, M. (2021) · arXiv (Cornell University) · Cited by 118 · see citing works
Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method
Feng, F., Huang, W., He, X., Xin, X., Wang, Q., & Chua, T. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462971 · Cited by 65 · see citing works
Learning Conjoint Attentions for Graph Neural NetsPreprint
He, T., Ong, Y.-S., & Bai, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03147 · Cited by 60 · see citing works
Meta-HGT: Metapath-aware HyperGraph Transformer for heterogeneous information network embedding
Liu, J., Song, L., Wang, G., & Shang, X. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.08.028 · Cited by 47 · see citing works
Self-citationYin, N., Shen, L., Xiong, H., Gu, B., Chen, C., Hua, X., Liu, S., & Luo, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331389 · Cited by 47 · see citing works
Exploiting Neighbor Effect: Conv-Agnostic GNN Framework for Graphs With Heterophily
Chen, J., Chen, S., Gao, J., Huang, Z., Zhang, J., & Pu, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3267902 · Cited by 47 · see citing works
Vision Transformers with Patch Diversification
Gong, C., Wang, D., Li, M., Chandra, V., & Liu, Q. (2021) · arXiv · doi:10.48550/arxiv.2104.12753 · Cited by 41 · see citing works
ProGraML: Graph-based Deep Learning for Program Optimization and AnalysisPreprint
Cummins, C., Fisches, Z. V., Ben-Nun, T., Hoefler, T., & Leather, H. (2020) · arXiv.org · doi:10.48550/arxiv.2003.10536 · Cited by 72 · see citing works
Multi-hop Attention Graph Neural NetworkPreprint
Wang, G., Ying, R., Huang, J., & Leskovec, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14332 · Cited by 38 · see citing works
Self-citationSkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks
Lu, W., Zhan, Y., Lin, B., Guan, Z., Liu, L., Yu, B., Zhao, W. Z., Yang, Y., & Tao, D. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3374701 · Cited by 31 · see citing works
A Heterogeneous Graph Neural Network With Attribute Enhancement and Structure-Aware Attention
Fan, S., Liu, G., & Li, J. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3239034 · Cited by 30 · see citing works
HGEN: Learning Hierarchical Heterogeneous Graph Encoding for Math Word Problem Solving
Zhang, Y., Zhou, G., Xie, Z., & Huang, J. X. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3145314 · Cited by 28 · see citing works
Self-citationFrom Cluster Assumption to Graph Convolution: Graph-Based Semi-Supervised Learning Revisited
Wang, Z., Ding, H., Pan, L., Li, J., Gong, Z., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3454710 · Cited by 30 · see citing works
Pathfinder Discovery Networks for Neural Message Passing
Rózemberczki, B., Englert, P. A. J., Kapoor, A., Blais, M., & Perozzi, B. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449882 · Cited by 27 · see citing works
Graph Decoupling Attention Markov Networks for Semisupervised Graph Node Classification
Chen, J., Chen, S., Bai, M., Pu, J., Zhang, J., & Gao, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3161453 · Cited by 26 · see citing works
Learning Cortical Parcellations Using Graph Neural Networks
Eschenburg, K. M., Grabowski, T. J., & Haynor, D. R. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.797500 · Cited by 26 · see citing works
Node classification oriented Adaptive Multichannel Heterogeneous Graph Neural Network
Li, Y., Jian, C., Zang, G., Song, C., & Yuan, X. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111618 · Cited by 25 · see citing works
AdaRisk: Risk-Adaptive Deep Reinforcement Learning for Vulnerable Nodes Detection
Li, F., Xu, Z., Cheng, D., & Wang, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3409869 · Cited by 23 · see citing works
Toward Unsupervised Graph Neural Network: Interactive Clustering and Embedding via Optimal Transport
Yang, L., Gu, J., Wang, C., Cao, X., Zhai, L., Jin, D., & Guo, Y. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00177 · Cited by 23 · see citing works
Distinctiveness oriented Positional Equilibrium for Point Cloud Registration
Min, T., Song, C., Kim, E.-S., & Shim, I. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00544 · Cited by 22 · see citing works
Adaptive Kernel Graph Neural Network
Ju, M., Hou, S., Fan, Y., Zhao, J., Ye, Y., & Zhao, L. (2022) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v36i6.20664 · Cited by 21 · see citing works
Pointer Graph NetworksPreprint
Veličković, P., Buesing, L., Overlan, M. C., Pascanu, R., Vinyals, O., & Blundell, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06380 · Cited by 18 · see citing works
An Embedding-Driven Multi-Hop Spatio-Temporal Attention Network for Traffic Prediction
Xue, R., Zhao, S., & Han, F. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3220915 · Cited by 15 · see citing works
Stabilizing and Enhancing Link Prediction through Deepened Graph Auto-Encoders
Wu, X., & Cheng, Q. S. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/498 · Cited by 19 · see citing works
Social-PatteRNN: Socially-Aware Trajectory Prediction Guided by Motion Patterns
Navarro, I., & Oh, J. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981486 · Cited by 14 · see citing works
ELECTRA-based graph network model for multi-hop question answering
Zhu, P., Yuan, Y., & Chen, L. (2023) · Journal of Intelligent Information Systems · doi:10.1007/s10844-023-00800-5 · Cited by 14 · see citing works
Domain-Adaptive Graph Attention-Supervised Network for Cross-Network Edge Classification
Shen, X. X., Shao, M., Pan, S., Yang, L. T., & Zhou, X. F. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3309632 · Cited by 13 · see citing works
Path reliability-based graph attention networks
Li, Y., Liang, S., & Jiang, Y. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.021 · Cited by 12 · see citing works
Can Graph Neural Networks Go Deeper Without Over-Smoothing? Yes, With a Randomized Path Exploration!
Bose, K., & Das, S. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3249255 · Cited by 12 · see citing works
Graph-CAT: Graph Co-Attention Networks via local and global attribute augmentations
Yang, L., Li, W., Guo, Y., & Gu, J. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.12.026 · Cited by 12 · see citing works
Adaptive Graph Diffusion Networks with Hop-wise AttentionPreprint
Sun, C., & Wu, G. (2020) · arXiv (Cornell University) · Cited by 12 · see citing works
Deng, Y., Bian, J., Wu, S., Lai, J., & Xie, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102711 · Cited by 12 · see citing works
Liu, X., He, Y., Tai, W., Xu, X., Zhou, F., & Luo, G. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010067 · Cited by 10 · see citing works
Fan, H., Zhong, Y., Zeng, G., & Sun, L. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01825-x · Cited by 9 · see citing works
Node Masking: Making Graph Neural Networks Generalize and Scale Better
Mishra, P., Piktus, A., Goossen, G., & Silvestri, F. (2020) · arXiv · doi:10.48550/arxiv.2001.07524 · Cited by 8 · see citing works
Guo, Y., Qiao, L., Yang, Z., Xiang, J., Feng, X., & Ma, H. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010104 · Cited by 8 · see citing works
Chen, Z., ESPOSITO, M. L., Xu, Z., & Zhou, W. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103374 · Cited by 7 · see citing works
Wang, H., Chen, J., Du, L., Fu, Q., Shi, H., & Song, X. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/257 · Cited by 7 · see citing works
Deepened Graph Auto-Encoders Help Stabilize and Enhance Link PredictionPreprint
Wu, X., & Cheng, Q. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11414 · Cited by 6 · see citing works
Subgraph-Based Attention Network for Multi-Hop Question Answering
Gong, C., Wei, Z., Wang, X., Wang, R., Li, Y., & Zhu, P. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650851 · Cited by 3 · see citing works
Sun, Q., He, Y., Yin, L., & Petrosian, O. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128883 · Cited by 3 · see citing works
PageRank Centrality based Graph Convolutional Networks for Semi-supervised Node Classification
Chen, L., Liu, J., Zhang, X., & Zhou, Q. (2021) · International Conference on Computer Science & Education (ICCSE) · doi:10.1109/iccse51940.2021.9569406 · Cited by 3 · see citing works
Node Embedding and Classification with Adaptive Structural Fingerprint
Zhu, Y., Wang, J., Zhang, J., & Zhang, K. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.073 · Cited by 3 · see citing works
Multi-Hop Diffusion-Based Graph Convolutional Networks
Bai, Y., Liu, S., & Tang, Y. (2022) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait56212.2022.10137919 · Cited by 2 · see citing works
PatchSkip: A lightweight technique for effectively alleviating over-smoothing in vision transformers
Zhao, J., Ye, X., Li, B., & Li, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128112 · Cited by 2 · see citing works
PBGAN: Path Based Graph Attention Network for Heterophily
Guan, X., Wang, D., Xiong, C., Li, S., & Chen, Y. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956658 · Cited by 2 · see citing works
Enhancing Emotion Detection with Sentiment Analysis InsightsPreprint
Parker, E., Nasir, W., & Benson, O. (2024) · Preprints.org · doi:10.20944/preprints202404.0059.v1 · Cited by 2 · see citing works
Baz, M. (2024) · Mathematics · doi:10.3390/math12213320 · Cited by 1 · see citing works
BaSFormer: A Balanced Sparsity Regularized Attention Network for Transformer
Jiang, S., Chen, Q., Xiang, Y., Pan, Y., & Wu, X. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3374062 · Cited by 1 · see citing works
Thai Question Text-To-SQL Parsing Using Transformer
Tungruethaipak, N., & Prom–on, S. (2024) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse61278.2024.10613642 · Cited by 1 · see citing works
Sandeep, S., & Sonnad, S. (2024) · International Journal of Image and Data Fusion · doi:10.1080/19479832.2024.2425710 · Cited by 2 · see citing works
Multi-hop Relational Graph Attention Network for Text-to-SQL Parsing
Hu, L., Shi, Y., Zhang, J., Wang, X., Li, H., & Kong, F. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191914
Multi-hop Attention GNN with Answer-Evidence Contrastive Loss for Multi-hop QA
Yang, N., & Yang, M. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191117
Leveraging Sentiment Lexicon in Sentiment DetectionPreprint
Johnson, A., DAVIS, E. S., Nasir, W., & Brown, M. H. S. (2024) · Preprints.org · doi:10.20944/preprints202404.0254.v1
Improve Graph Convolutional Network Performance via Incorporating Message Filter
Qiu, D., Cai, K., Lei, S., & Wu, X. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00042
Zephyr: A High-Performance Framework for Graph Attention Networks on Heterogeneous Data
Ding, W., Liu, M., Cai, Y., Chen, M., Yan, Z., & Wang, M. (2024) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom63139.2024.00135
ADP-Net: a hierarchical attention-diffusion-prediction framework for human trajectory prediction
Zhang, Z., Xiao, S., & Yu, Z. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1690704
How Attentive are Graph Attention Networks?
Brody, S., Alon, U., & Yahav, E. (2021) · arXiv · doi:10.48550/arxiv.2105.14491 · Cited by 789 · see citing works
- Authors
- Wang, G., Ying, R., Huang, J., & Leskovec, J.
- Affiliations
- Stanford University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1910.11945
- OpenAlex
- W2981525344
- Cited by
- 61
- References
- 40
- Citations by year
- 2025: 1 · 2024: 18 · 2023: 9 · 2022: 13 · 2021: 12 · 2020: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Graph Theory and Algorithms
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Keywords
- graph attention networks, over-smoothing, overfitting, graph representation learning, feature aggregation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
