CoChat
Sign Up
PreprintCited by 10,639Advanced Graph Neural NetworksComplex Network Analysis TechniquesRecommender Systems and Techniques

Inductive Representation Learning on Large Graphs

Open access
Hamilton, William L., Ying, Rex, Leskovec, Jure·2017·arXiv
DOI 10.48550/arxiv.1706.02216arXiv 1706.02216
Open this paper in CoChat — free accountOpen PDF

Abstract

Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1706.02216

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [2][2] S. Arora, Y. Liang, and T. Ma. A simple but tough-to-beat baseline for sentence embeddings. In ICLR , 2017. -
  2. [3][3] A. R. Benson, D. F. Gleich, and J. Leskovec. Higher-order organization of complex networks. Science , 353(6295):163-166, 2016. -
  3. [4][4] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun. Spectral networks and locally connected networks on graphs. In ICLR , 2014. -
  4. [5][5] S. Cao, W. Lu, and Q. Xu. Grarep: Learning graph representations with global structural information. In KDD , 2015. -
  5. [6][6] J. Chen and J. Zhu. Stochastic training of graph convolutional networks. arXiv preprint arXiv:1710.10568 , 2017. - · arXiv
  6. [7][7] H. Dai, B. Dai, and L. Song. Discriminative embeddings of latent variable models for structured data. In ICML , 2016. -
  7. [8][8] M. Defferrard, X. Bresson, and P. Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS , 2016. -
  8. [9][9] D. K. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams. Convolutional networks on graphs for learning molecular fingerprints. In NIPS , 2015. -
  9. [10][10] M. Gori, G. Monfardini, and F. Scarselli. A new model for learning in graph domains. In IEEE International Joint Conference on Neural Networks , volume 2, pages 729-734, 2005. -
  10. [12][12] W. L. Hamilton, J. Leskovec, and D. Jurafsky. Diachronic word embeddings reveal statistical laws of semantic change. In ACL , 2016. -
  11. [15][15] K. Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks , 4(2):251-257, 1991. -
  12. [16][16] D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR , 2015. -
  13. [17][17] T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR , 2016. -
  14. [18][18] T. N. Kipf and M. Welling. Variational graph auto-encoders. In NIPS Workshop on Bayesian Deep Learning , 2016. -
  15. [19][19] J. B. Kruskal. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika , 29(1):1-27, 1964. -
  16. [20][20] O. Levy and Y. Goldberg. Neural word embedding as implicit matrix factorization. In NIPS , 2014. -
  17. [21][21] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. Gated graph sequence neural networks. In ICLR , 2015. -
  18. [22][22] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS , 2013. -
  19. [23][23] A. Y. Ng, M. I. Jordan, Y. Weiss, et al. On spectral clustering: Analysis and an algorithm. In NIPS , 2001. -
  20. [24][24] M. Niepert, M. Ahmed, and K. Kutzkov. Learning convolutional neural networks for graphs. In ICML , 2016. -
  21. [26][26] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011. -
  22. [27][27] J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP , 2014. -
  23. [28][28] B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. In KDD , 2014. -
  24. [29][29] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR , 2017. -
  25. [30][30] R. ˇ Reh˚ uˇ rek and P. Sojka. Software Framework for Topic Modelling with Large Corpora. In LREC , 2010. -
  26. [31][31] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Transactions on Neural Networks , 20(1):61-80, 2009. -
  27. [32][32] N. Shervashidze, P. Schweitzer, E. J. v. Leeuwen, K. Mehlhorn, and K. M. Borgwardt. Weisfeilerlehman graph kernels. Journal of Machine Learning Research , 12:2539-2561, 2011. -
  28. [33][33] S. Siegal. Nonparametric statistics for the behavioral sciences . McGraw-hill, 1956. -
  29. [36][36] D. Wang, P. Cui, and W. Zhu. Structural deep network embedding. In KDD , 2016. -
  30. [37][37] X. Wang, P. Cui, J. Wang, J. Pei, W. Zhu, and S. Yang. Community preserving network embedding. In AAAI , 2017. -
  31. [38][38] D. J. Watts and S. H. Strogatz. Collective dynamics of 'small-world' networks. Nature , 393(6684):440-442, 1998. -
  32. [40][40] Z. Yang, W. Cohen, and R. Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In ICML , 2016. -
  33. [41][41] M. Zitnik and J. Leskovec. Predicting multicellular function through multi-layer tissue networks. Bioinformatics , 33(14):190-198, 2017.

Cited by

Sorted by citations

Authors
Hamilton, W. L., Ying, R., & Leskovec, J.
Affiliations
Stanford University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
Volume
30
Pages
1024-1034
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1706.02216
arXiv
1706.02216
OpenAlex
W2624431344
Cited by
10,639
References
4
Citations by year
2026: 86 · 2025: 1242 · 2024: 2142 · 2023: 1920 · 2022: 1632 · 2021: 1796 · 2020: 1112 · 2019: 549 · 2018: 143 · 2017: 15
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • neural information processing systems
Funders
  • National Science Foundation (1149837)
  • European Commission (882500, HORIZON2020)
  • Royal College of Nursing, Australia
  • Norges Forskningsråd
  • Defense Advanced Research Projects Agency
  • Natural Sciences and Engineering Research Council of Canada
  • Fundação para a Ciência e a Tecnologia (UIDB/00127/2020, UI/BD/153571/2022)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
Keywords
node embeddings, large-scale graphs, GraphSAGE, node attributes, citation networks, Reddit posts, protein-protein interaction, multi-graph learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review