Graph Attention Networks
Open accessAbstract
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GAT's superior performance in processing large amounts of data in real-time has led to it becoming one of the most popular methods for designing aggregation functions in most modern GNNs.
“Although GNNs impose no constraints on the form of the aggregation function, the superior performance of the Graph Attention Network (GAT) [90] in processing large amounts of data in real-time has led to it becoming one of the most popular methods for designing aggregation functions in most modern GNNs.”
From TVGeAN: Tensor Visibility Graph-Enhanced Attention Network for Versatile Multivariant Time Series Learning Tasks (2024) · cites this paper as [90]
The Results (p. 8) report strong benchmark performance, while Comparisons to Related Work (pp. 4–5) describes an efficient, parallelizable attention layer and a version that can run on larger graph datasets. The paper does not establish superior real-time processing of large amounts of data or show that GAT became a popular aggregation method in modern GNNs because of such performance. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe static Pearson-correlation-based adjacency matrix assumed fixed spatial relationships, failing to account for dynamic atmospheric processes such as shifting jet streams or evolving ENSO phases.
“The static Pearson-correlation-based adjacency matrix assumed fixed spatial relationships, failing to account for the dynamic nature of atmospheric processes, such as shifting jet streams or evolving El Ni˜ no-Southern Oscillation (ENSO) phases [26, 27].”
From Advancing Heatwave Forecasting via Distribution Informed-Graph Neural Networks (DI-GNNs): Integrating Extreme Value Theory with GNNs (2024) · cites this paper as [27]
The graph attentional layer section (p. 3) describes attention over a graph’s neighbors, and the Introduction (p. 2) discusses dependence on graph structure. Neither addresses a Pearson-correlation adjacency matrix or whether fixed spatial relationships fail to capture changing atmospheric processes. Searches for Pearson correlation, jet streams, ENSO, and changing adjacency found no supporting passage. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625 · see citing works
(2014) · Cited by 33,419 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · see citing works
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639 · see citing works
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123 · see citing works
Advances In Deep Learning On Graphs (Gsp'18 Workshop)Preprint
Defferrard, M., Bresson, X., & Vandergheynst, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.09375 · Cited by 6,912 · see citing works
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513 · see citing works
Spectral Networks and Locally Connected Networks on Graphs
Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6203 · Cited by 3,983 · see citing works
Gated Graph Sequence Neural NetworksPreprint
Li, Y., Tarlow, D., Brockschmidt, M., & Zemel, R. (2015) · International Conference on Learning Representations · doi:10.48550/arxiv.1511.05493 · Cited by 3,735 · see citing works
(2006) · Cited by 3,624 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Self-citationLearning Convolutional Neural Networks for GraphsPreprint
Niepert, M., Ahmed, M., & Kutzkov, K. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1605.05273 · Cited by 2,311 · see citing works
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs
Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., & Bronstein, M. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.576 · Cited by 1,820 · see citing works
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
Jégou, S., Drozdzal, M., Vázquez, D., Romero, A., & Bengio, Y. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.156 · Cited by 1,762 · see citing works
Self-citationRevisiting Semi-Supervised Learning with Graph EmbeddingsPreprint
Yang, Z., Cohen, W. W., & Salakhutdinov, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08861 · Cited by 1,366 · see citing works
A simple neural network module for relational reasoning
Santoro, A., Raposo, D., Barrett, D. G. T., Malinowski, M., Pascanu, R., Battaglia, P., & Lillicrap, T. (2017) · arXiv · doi:10.48550/arxiv.1706.01427 · Cited by 1,281 · see citing works
Deep Convolutional Networks on Graph-Structured Data
Henaff, M., Bruna, J., & LeCun, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.05163 · Cited by 1,255 · see citing works
Long Short-Term Memory-Networks for Machine Reading
Cheng, J.-P., Dong, L., & Lapata, M. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1053 · Cited by 1,200 · see citing works
Diffusion-Convolutional Neural Networks
Atwood, J. R., & Towsley, D. (2016) · neural information processing systems · Cited by 566 · see citing works
Predicting multicellular function through multi-layer tissue networks
Žitnik, M., & Leskovec, J. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx252 · Cited by 515 · see citing works
A general framework for adaptive processing of data structures
Frasconi, P., Gori, M., & Sperduti, A. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.712151 · Cited by 481 · see citing works
One-Shot Imitation LearningPreprint
Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, O. J., Schneider, J., Sutskever, I., Abbeel, P., & Zaremba, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07326 · Cited by 449 · see citing works
A Convolutional Encoder Model for Neural Machine TranslationPreprint
Gehring, J., Auli, M., Grangier, D., & Dauphin, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02344 · Cited by 52 · see citing works
Flagged first, then by citations
3 of the 31 references couldn’t be matched to a known paper.
Research works printed but not linked: 16
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations (ICLR) , 2015.
- Micha¨ el Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems , pp. 3844-3852, 2016.
- Misha Denil, Sergio G´ omez Colmenarejo, Serkan Cabi, David Saxton, and Nando de Freitas. Programmable agents. arXiv preprint arXiv:1706.06383 , 2017. · arXiv
- David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Al´ an Aspuru-Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems , pp. 2224-2232, 2015.
- Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pp. 249-256, 2010.
- Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In IEEE International Joint Conference on Neural Networks , pp. 729734, 2005.
- Yedid Hoshen. Vain: Attentional multi-agent predictive modeling. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30 , pp. 26982708. Curran Associates, Inc., 2017. URL http://papers.nips.cc/paper/ 6863-vain-attentional-multi-agent-predictive-modeling.pdf .
- Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR) , 2017.
- Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130 , 2017. · arXiv
- Qing Lu and Lise Getoor. Link-based classification. In Proceedings of the 20th International Conference on Machine Learning (ICML-03) , pp. 496-503, 2003.
- Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks , 20(1):61-80, 2009.
- Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine , 29(3):93, 2008.
- A. Sperduti and A. Starita. Supervised neural networks for the classification of structures. Trans. Neur. Netw. , 8(3):714-735, May 1997. ISSN 1045-9227. doi: 10.1109/72.572108. · DOI · Paper page
- Jason Weston, Fr´ ed´ eric Ratle, Hossein Mobahi, and Ronan Collobert. Deep learning via semisupervised embedding. In Neural Networks: Tricks of the Trade , pp. 639-655. Springer, 2012.
- Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. CoRR , abs/1410.3916, 2014. URL http://arxiv.org/abs/1410.3916 . · arXiv
- Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03) , pp. 912-919, 2003.
Cited by
Sorted by citations
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000 · see citing works
Dynamic Graph CNN for Learning on Point Clouds
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019) · ACM Transactions on Graphics · doi:10.1145/3326362 · Cited by 7,199 · see citing works
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880 · see citing works
Formal Semantics for Kolmogorov-Arnold Network Representations of Operational GamesPreprint
Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00826 · Cited by 4,904 · see citing works
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401063 · Cited by 5,083 · see citing works
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · see citing works
A review on the attention mechanism of deep learning
Niu, Z., Zhong, G., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.091 · Cited by 3,379 · see citing works
Graph WaveNet for Deep Spatial-Temporal Graph Modeling
Wu, Z., Pan, S., Long, G., Jiang, J. B., & Zhang, C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/264 · Cited by 3,158 · see citing works
Heterogeneous Graph Attention Network
Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., & Yu, P. S. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313562 · Cited by 3,191 · see citing works
Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting
Guo, S., Lin, Y., Feng, N., Song, C., & Wan, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301922 · Cited by 3,123 · see citing works
SuperGlue: Learning Feature Matching With Graph Neural Networks
Sarlin, P.-E., DeTone, D., Malisiewicz, T., & Rabinovich, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00499 · Cited by 3,002 · see citing works
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. (2020) · UTS ePRESS (University of Technology Sydney) · Cited by 3,174 · see citing works
Fast Graph Representation Learning with PyTorch GeometricPreprint
Fey, M., & Lenssen, J. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02428 · Cited by 3,402 · see citing works
Self-Attention with Relative Position Representations
Shaw, P., Uszkoreit, J., & Vaswani, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2074 · Cited by 2,505 · see citing works
GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond
Cao, Y., Xu, J., Lin, S., Wei, F., & Hu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00246 · Cited by 2,307 · see citing works
Wang, X., He, X., Cao, Y., Liu, M., & Chua, T. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330989 · Cited by 2,274 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Simplifying Graph Convolutional NetworksPreprint
Wu, F., Zhang, T., Junior, A. H. D. S., Fifty, C., Yu, T., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07153 · Cited by 2,251 · see citing works
Graph Convolutional Networks for Text Classification
Yao, L., Mao, C., & Luo, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017370 · Cited by 2,171 · see citing works
Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., de Melo, G., Gutiérrez, C., Kirrane, S., Gayo, J. E. L., Roberto Navigli, A., Neumaier, S., et al. (2021) · ACM Computing Surveys · doi:10.1145/3447772 · Cited by 2,371 · see citing works
Feng, Y., You, H., Zhang, Z., Ji, R., & Gao, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013558 · Cited by 1,895 · see citing works
Graph convolutional networks: a comprehensive review
Zhang, S., Tong, H., Xu, J., & Maciejewski, R. (2019) · Computational Social Networks · doi:10.1186/s40649-019-0069-y · Cited by 1,888 · see citing works
GMAN: A Graph Multi-Attention Network for Traffic Prediction
Zheng, C., Fan, X., Wang, C., & Qi, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5477 · Cited by 1,849 · see citing works
Song, C., Lin, Y.-F., Guo, S., & Wan, H.-Y. (2020) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5438 · Cited by 1,897 · see citing works
Deep Learning on Graphs: A Survey
Zhang, Z., Cui, P., & Zhu, W. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981333 · Cited by 1,729 · see citing works
Heterogeneous Graph Neural Network
Zhang, C., Song, D., Huang, C., Swami, A., & Chawla, N. V. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330961 · Cited by 1,591 · see citing works
Deep Learning--based Text Classification
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad-Khasmakhi, N., Asgari-Chenaghlu, M., & Gao, J. (2021) · ACM Computing Surveys · doi:10.1145/3439726 · Cited by 1,967 · see citing works
DeepGCNs: Can GCNs Go As Deep As CNNs?
Li, G., Muller, M., Thabet, A., & Ghanem, B. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00936 · Cited by 1,540 · see citing works
Heterogeneous Graph Transformer
Hu, Z., Dong, Y., Wang, K., & Sun, Y. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380027 · Cited by 1,646 · see citing works
Graph Neural Networks in Recommender Systems: A Survey
Wu, S., Sun, F., Zhang, W., Xie, X. H., & Cui, B. (2022) · ACM Computing Surveys · doi:10.1145/3535101 · Cited by 1,466 · see citing works
Graph neural network for traffic forecasting: A survey
Jiang, W., & Luo, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117921 · Cited by 1,391 · see citing works
Graph Neural Network-Based Anomaly Detection in Multivariate Time Series
Deng, A., & Hooi, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16523 · Cited by 1,413 · see citing works
Graph Contrastive Learning with AugmentationsPreprint
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., & Shen, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13902 · Cited by 1,400 · see citing works
Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition
Liu, Z., Zhang, H., Chen, Z., Wang, Z., & Ouyang, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00022 · Cited by 1,307 · see citing works
Representation Learning on Graphs with Jumping Knowledge NetworksPreprint
Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K., & Jegelka, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03536 · Cited by 1,379 · see citing works
EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs
Pareja, A., Domeniconi, G., Chen, J., Ma, T., Suzumura, T., Kanezashi, H., Kaler, T., Schardl, T. B., & Leiserson, C. E. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5984 · Cited by 128 · see citing works
Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
Chiang, W.-L., Liu, X., Si, S., Li, Y., Bengio, S., & Hsieh, C.-J. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330925 · Cited by 1,331 · see citing works
Open Graph Benchmark: Datasets for Machine Learning on GraphsPreprint
Hu, W., Fey, M., Žitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., & Leskovec, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00687 · Cited by 1,439 · see citing works
Meshed-Memory Transformer for Image Captioning
Cornia, M., Stefanini, M., Baraldi, L., & Cucchiara, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01059 · Cited by 1,259 · see citing works
Recent advances and applications of deep learning methods in materials science
Choudhary, K., DeCost, B., Chen, C., Jain, A., Tavazza, F., Cohn, R., Park, C. W., Choudhary, A., Agrawal, A., Billinge, S. J. L., et al. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00734-6 · Cited by 1,172 · see citing works
Graph Contrastive Learning with Adaptive Augmentation
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., & Wang, L. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449802 · Cited by 1,192 · see citing works
Deep Closest Point: Learning Representations for Point Cloud Registration
Wang, Y., & Solomon, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00362 · Cited by 1,151 · see citing works
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
Chen, Y.-X., Zhang, Z.-Q., Yuan, C.-F., Li, B., Deng, Y., & Hu, W.-M. (2021) · IEEE International Conference on Computer Vision · doi:10.1109/iccv48922.2021.01311 · Cited by 1,057 · see citing works
Xiong, Z., Wang, D., Liu, X., Zhong, F., Wan, X., Li, X., Li, Z. S., Luo, X., Chen, K., Jiang, H., et al. (2019) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.9b00959 · Cited by 1,083 · see citing works
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063 · see citing works
MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding
Fu, X., Zhang, J., Meng, Z., & King, I. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380297 · Cited by 1,056 · see citing works
Adaptive Graph Convolutional Recurrent Network for Traffic ForecastingPreprint
Bai, L., Yao, L., Li, C., Wang, X., & Wang, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02842 · Cited by 1,013 · see citing works
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01154 · Cited by 1,011 · see citing works
Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting
Li, M., & Zhu, Z. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16542 · Cited by 1,005 · see citing works
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994 · see citing works
A Survey on Deep Semi-Supervised Learning
Yang, X., Song, Z., King, I., & Xu, Z. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3220219 · Cited by 981 · see citing works
Structure-based protein function prediction using graph convolutional networks
Gligorijević, V., Renfrew, P. D., Kościółek, T., Leman, J. K., Berenberg, D., Vatanen, T., Chandler, C., Taylor, B. C., Fisk, I., Vlamakis, H. C., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-23303-9 · Cited by 913 · see citing works
PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing
Zhao, H., Jiang, L., Fu, C., & Jia, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00571 · Cited by 898 · see citing works
Graph R-CNN for Scene Graph Generation
Yang, J., Lu, J., Lee, S., Batra, D., & Parikh, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_41 · Cited by 889 · see citing works
Graph neural networks for materials science and chemistry
Reiser, P., Neubert, M., Eberhard, A., Torresi, L., Zhou, C., Shao, C., Metni, H., van Hoesel, C., Schopmans, H., Sommer, T., et al. (2022) · Communications Materials · doi:10.1038/s43246-022-00315-6 · Cited by 903 · see citing works
Contrastive Multi-View Representation Learning on GraphsPreprint
Hassani, K., & Khasahmadi, A. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05582 · Cited by 883 · see citing works
A Comprehensive Survey on Graph Anomaly Detection With Deep Learning
Ma, X., Wu, J. X., Xue, S., Yang, J. Y., Zhou, C., Sheng, Q. Z., Xiong, H., & Akoglu, L. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3118815 · Cited by 883 · see citing works
Multivariate Time-Series Anomaly Detection via Graph Attention Network
Zhao, H., Wang, Y., Duan, J., Huang, C., Cao, D., Tong, Y., Xu, B., Bai, J., Tong, J., & Zhang, Q. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00093 · Cited by 850 · see citing works
BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis
Li, X., Zhou, Y., Dvornek, N. C., Zhang, M., Gao, S., Zhuang, J., Scheinost, D., Staib, L. H., Ventola, P. E., & Duncan, J. S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102233 · Cited by 845 · see citing works
Hypergraph convolution and hypergraph attention
Bai, S., Zhang, F., & Torr, P. H. S. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107637 · Cited by 835 · see citing works
Hierarchical Graph Representation Learning with Differentiable PoolingPreprint
Ying, R., You, J., Morris, C., Ren, X., Hamilton, W. L., & Leskovec, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08804 · Cited by 798 · see citing works
Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
Shi, Y., Huang, Z., Feng, S., Zhong, H., Wang, W., & Sun, Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/214 · Cited by 870 · see citing works
Graph Attention Convolution for Point Cloud Semantic Segmentation
Wang, L., Huang, Y., Hou, Y., Zhang, S., & Shan, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01054 · Cited by 782 · see citing works
An Attentive Survey of Attention Models
Chaudhari, S., Mithal, V., Polatkan, G., & Ramanath, R. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3465055 · Cited by 819 · see citing works
Reinforcement learning for combinatorial optimization: A survey
Mazyavkina, N., Sviridov, S., Ivanov, S., & Burnaev, E. V. (2021) · Computers & Operations Research · doi:10.1016/j.cor.2021.105400 · Cited by 762 · see citing works
MeshCNN: a network with an edge
Hanocka, R., Hertz, A., Fish, N., Giryes, R., Fleishman, S., & Cohen-Or, D. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322959 · Cited by 742 · see citing works
Wei, Y., Wang, X., Nie, L., He, X., Hong, R., & Chua, T. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351034 · Cited by 735 · see citing works
Relational Graph Attention Network for Aspect-based Sentiment Analysis
Wang, K., Shen, W., Yang, Y., Quan, X., & Wang, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.295 · Cited by 749 · see citing works
A General Survey on Attention Mechanisms in Deep Learning
Brauwers, G., & Frăsincar, F. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3126456 · Cited by 722 · see citing works
STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction
Huang, Y., Bi, H., Li, Z., Mao, T., & Wang, Z. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00637 · Cited by 697 · see citing works
Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems
Wang, H., Zhang, F., Zhang, M., Leskovec, J., Zhao, M., Li, W., & Wang, Z. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330836 · Cited by 685 · see citing works
Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review
Mozaffari, S., Al-Jarrah, O. Y., Dianati, M., Jennings, P., & Mouzakitis, A. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3012034 · Cited by 679 · see citing works
Learning Attention-based Embeddings for Relation Prediction in \nKnowledge Graphs
Nathani, D., Chauhan, J., Sharma, C., & Kaul, M. (2019) · Research Archive of Indian Institute of Technology Hyderabad (Indian Institute of Technology Hyderabad) · Cited by 665 · see citing works
Chen, L., Wu, L., Hong, R., Zhang, K., & Wang, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5330 · Cited by 663 · see citing works
Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks
Wang, Z., Lv, Q., Lan, X., & Zhang, Y. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1032 · Cited by 662 · see citing works
Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters
Dou, Y., Liu, Z., Sun, L., Deng, Y., Peng, H. M., & Yu, P. S. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411903 · Cited by 677 · see citing works
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
Zhu, X., Zhou, H., Wang, T., Hong, F., Ma, Y., Li, W., Li, H., & Lin, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00981 · Cited by 647 · see citing works
Deep Learning for Time Series Anomaly Detection: A Survey
Darban, Z. Z., Webb, G. I., Pan, S., Aggarwal, C. C., & Salehi, M. (2024) · ACM Computing Surveys · doi:10.1145/3691338 · Cited by 743 · see citing works
Attributed Graph Clustering: A Deep Attentional Embedding Approach
Wang, C., Pan, S., Hu, R., Long, G., Jiang, J. B., & Zhang, C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/509 · Cited by 619 · see citing works
Learning Intents behind Interactions with Knowledge Graph for Recommendation
Wang, X., Huang, T., Wang, D., Yuan, Y., Liu, Z., He, X., & Chua, T. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450133 · Cited by 619 · see citing works
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review
Li, Y., Ma, L., Zhong, Z., Liu, F., Chapman, M. A., Cao, D., & Li, J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3015992 · Cited by 619 · see citing works
Geometry-enhanced molecular representation learning for property prediction
Fang, X., Liu, L., Lei, J., He, D., Zhang, S., Zhou, J., Wang, F., Wu, H. R., & Wang, H. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-021-00438-4 · Cited by 606 · see citing works
Explainability in Graph Neural Networks: A Taxonomic Survey
Yuan, H., Yu, H., Gui, S., & Ji, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204236 · Cited by 640 · see citing works
Pitfalls of Graph Neural Network EvaluationPreprint
Shchur, O., Mumme, M., Bojchevski, A., & Günnemann, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05868 · Cited by 1,031 · see citing works
GraphSAINT: Graph Sampling Based Inductive Learning MethodPreprint
Zeng, H., Zhou, H., Srivastava, A., Kannan, R., & Prasanna, V. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04931 · Cited by 573 · see citing works
Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-identification
Ye, M., Shen, J., Crandall, D., Shao, L., & Luo, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58520-4_14 · Cited by 571 · see citing works
Jin, M., Koh, H. Y., Wen, Q., Zambon, D., Alippi, C., Webb, G. I., King, I., & Pan, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3443141 · Cited by 567 · see citing works
Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solution
Li, F., Feng, J., Yan, H., Jin, G., Yang, F., Sun, F., Jin, D., & Li, Y. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3532611 · Cited by 585 · see citing works
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
Wang, B., Shin, R., Liu, X., Polozov, O., & Richardson, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.677 · Cited by 606 · see citing works
DeepInf: Social Influence Prediction with Deep Learning
Qiu, J., Tang, J., Ma, H., Dong, Y., Wang, K., & Tang, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220077 · Cited by 587 · see citing works
Edge-Labeling Graph Neural Network for Few-Shot Learning
Kim, J., Kim, T., Kim, S., & Yoo, C. D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00010 · Cited by 562 · see citing works
Attention Guided Graph Convolutional Networks for Relation Extraction
Guo, Z., Zhang, Y., & Lu, W. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1024 · Cited by 552 · see citing works
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
Yasunaga, M., Ren, H., Bosselut, A., Liang, P., & Leskovec, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.45 · Cited by 620 · see citing works
Graph Self-Supervised Learning: A Survey
Liu, Y., Jin, M., Pan, S., Zhou, C. D., Zheng, Y., Xia, F., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3172903 · Cited by 564 · see citing works
Towards Deeper Graph Neural Networks
Liu, M., Gao, H., & Ji, S. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403076 · Cited by 553 · see citing works
Guo, D., Shao, Y., Cui, Y., Wang, Z., Zhang, L., & Shen, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00942 · Cited by 537 · see citing works
DepGraph: Towards Any Structural Pruning
Fang, G., Ma, X., Song, M., Mi, M. B., & Wang, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01544 · Cited by 538 · see citing works
Learning Trajectory Dependencies for Human Motion Prediction
Mao, W., Liu, M.-M., Salzmann, M., & Li, H. (2019) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2019.00958 · Cited by 591 · see citing works
Machine learning for a sustainable energy future
Yao, Z., Lum, Y., Johnston, A. K., Mejía-Mendoza, L. M., Zhou, X., Wen, Y., Aspuru‐Guzik, A., Sargent, E. H., & Seh, Z. W. (2022) · Nature Reviews Materials · doi:10.1038/s41578-022-00490-5 · Cited by 541 · see citing works
- Authors
- Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y.
- Affiliations
- University of Cambridge (GB)
- HEC Montréal (CA)
- Universitat Autònoma de Barcelona (ES)
- Universitat Politècnica de Catalunya (ES)
- Université de Montréal (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1710.10903
- OpenAlex
- W2766453196
- Cited by
- 12,107
- References
- 31
- Citations by year
- 2026: 218 · 2025: 1367 · 2024: 2226 · 2023: 2274 · 2022: 1792 · 2021: 2299 · 2020: 1341 · 2019: 502 · 2018: 85 · 2017: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- arXiv (Cornell University)
- Apollo
- Funders
- Nvidia
- Canadian Institute for Advanced Research
- Compute Canada
- European Commission (634821)
- Canada Research Chairs
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Graph Theory and Algorithms
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Keywords
- graph attention networks, graph neural networks, self-attention, graph-structured data, inductive learning, transductive learning, citation networks, protein-protein interaction network, spectral graph neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
