CoChat
Sign Up
PreprintCited by 12,107Advanced Graph Neural NetworksGraph Theory and AlgorithmsComplex Network Analysis Techniques

The DOI matches its DataCite record; 28 of the 31 references it cites were found in a registry or an index.

Graph Attention Networks

Open access
Veličković, Petar, Cucurull, Guillem, Casanova, Arantxa, Romero, Adriana, Liò, Pietro, Bengio, Yoshua·2017·arXiv
DOI 10.48550/arxiv.1710.10903arXiv 1710.10903
Open this paper in CoChat — free accountOpen PDF

Abstract

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1710.10903

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 31 references

Research works printed but not linked: 16

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations (ICLR) , 2015.
  2. Micha¨ el Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems , pp. 3844-3852, 2016.
  3. Misha Denil, Sergio G´ omez Colmenarejo, Serkan Cabi, David Saxton, and Nando de Freitas. Programmable agents. arXiv preprint arXiv:1706.06383 , 2017. · arXiv
  4. David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Al´ an Aspuru-Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems , pp. 2224-2232, 2015.
  5. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pp. 249-256, 2010.
  6. Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In IEEE International Joint Conference on Neural Networks , pp. 729734, 2005.
  7. Yedid Hoshen. Vain: Attentional multi-agent predictive modeling. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30 , pp. 26982708. Curran Associates, Inc., 2017. URL http://papers.nips.cc/paper/ 6863-vain-attentional-multi-agent-predictive-modeling.pdf .
  8. Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR) , 2017.
  9. Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130 , 2017. · arXiv
  10. Qing Lu and Lise Getoor. Link-based classification. In Proceedings of the 20th International Conference on Machine Learning (ICML-03) , pp. 496-503, 2003.
  11. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks , 20(1):61-80, 2009.
  12. Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine , 29(3):93, 2008.
  13. A. Sperduti and A. Starita. Supervised neural networks for the classification of structures. Trans. Neur. Netw. , 8(3):714-735, May 1997. ISSN 1045-9227. doi: 10.1109/72.572108. · DOI · Paper page
  14. Jason Weston, Fr´ ed´ eric Ratle, Hossein Mobahi, and Ronan Collobert. Deep learning via semisupervised embedding. In Neural Networks: Tricks of the Trade , pp. 639-655. Springer, 2012.
  15. Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. CoRR , abs/1410.3916, 2014. URL http://arxiv.org/abs/1410.3916 . · arXiv
  16. Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03) , pp. 912-919, 2003.

Cited by

Sorted by citations

Items 201–300 of 1,010

Related

See all

Metadata

Authors
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y.
Affiliations
  • University of Cambridge (GB)
  • HEC Montréal (CA)
  • Universitat Autònoma de Barcelona (ES)
  • Universitat Politècnica de Catalunya (ES)
  • Université de Montréal (CA)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1710.10903
arXiv
1710.10903
OpenAlex
W2766453196
Cited by
12,107
References
31
Citations by year
2026: 218 · 2025: 1367 · 2024: 2226 · 2023: 2274 · 2022: 1792 · 2021: 2299 · 2020: 1341 · 2019: 502 · 2018: 85 · 2017: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • Apollo (University of Cambridge) (submittedVersion)
  • arXiv (Cornell University)
  • Apollo
Funders
  • Nvidia
  • Canadian Institute for Advanced Research
  • Compute Canada
  • European Commission (634821)
  • Canada Research Chairs
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Graph Theory and Algorithms
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
Keywords
graph attention networks, graph neural networks, self-attention, graph-structured data, inductive learning, transductive learning, citation networks, protein-protein interaction network, spectral graph neural networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review