CoChat
Sign Up
Cited by 3,733Recommender Systems and TechniquesAdvanced Graph Neural NetworksComplex Network Analysis Techniques

Graph Convolutional Neural Networks for Web-Scale Recommender Systems

Open access
Ying, Rex, He, Ruining, Chen, Kaifeng, Eksombatchai, Pong, Hamilton, William L., Leskovec, Jure·2018·Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
DOI 10.1145/3219819.3219890
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains an unsolved challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. Overall, we can train on and embed graphs that are four orders of magnitude larger than typical GCN implementations. We show how GCN embeddings can be used to make high-quality recommendations in various settings at Pinterest, which has a massive underlying graph with 3 billion nodes representing pins and boards, and 17 billion edges. According to offline metrics, user studies, as well as A/B tests, our approach generates higher-quality recommendations than comparable deep learning based systems. To our knowledge, this is by far the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.

AI snapshot

Written from the paper’s full text.

TL;DR

PinSage is a random-walk-based Graph Convolutional Network deployed at Pinterest that scales graph convolutions to a graph with 3 billion nodes and 18 billion edges using localized random-walk neighborhoods, importance pooling, curriculum training, and MapReduce inference. It reports better offline, human-evaluated, and production A/B recommendation performance than content-based and graph-based baselines.

Key findings

  • On related-pin offline evaluation, PinSage achieves 67% hit-rate and 0.59 MRR, versus the best baseline Combined at 27% hit-rate and 0.37 MRR; the paper reports 40% absolute (150% relative) hit-rate and 22% absolute (60% relative) MRR improvement over the best baseline.
  • In head-to-head user studies, PinSage recommendations are preferred about 60% of the time among items with an opinion: 56.5% vs. Visual, 72.5% vs. Annotation, 60.0% vs. Combined, and 62.4% vs. Pixie.
  • In production A/B tests on homefeed, PinSage gives 10–30% improvements in repin rate over Annotation and Visual embedding recommendations; the abstract/introduction also state 30% to 100% improvements in user engagement across various settings.
  • Importance pooling gives a 46% offline performance gain and curriculum training gives a 12% gain; MapReduce inference for 3 billion items completes in less than 24 hours.

Methods

  • Model: PinSage uses localized graph convolutions; neighborhoods are the top T nodes by L1-normalized random-walk visit counts (importance-based), with importance pooling; experiments use K=2 layers, hidden size 2048, embedding dimension 1024, and features concatenating 4,096-d visual embeddings, 256-d text embeddings, and log degree.
  • Training: supervised max-margin ranking loss with 1.2 billion positive pairs, 500 shared negative items per minibatch, 6 hard negatives per pin, totaling 7.5 billion training examples; curriculum training adds harder negatives over epochs; training uses a subgraph with 20% of boards and 70% of labeled examples.
  • Evaluation/deployment: offline hit-rate@500 and MRR on related-pin recommendation, controlled user studies, and production A/B tests on homefeed repin rate; MapReduce inference generates embeddings for the full graph.

Limitations

  • Stated: offline comparisons omit Pixie because its production use generated the labeled pairs; other GCN-based recommender systems are not directly comparable because they cannot scale to Pinterest-size data; other deep learning baselines from the literature are not compared due to scale.
  • Stated: training on a subgraph containing 20% of boards and 70% of labeled examples had negligible impact on final performance, but this remains a design choice rather than full-graph training.
  • Evident from the text: evaluation is on Pinterest proprietary data and production settings, with no public benchmark; A/B and user-study results are platform-specific and may not generalize.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3219819.3219890

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 14 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § 1 INTRODUCTION · p. 2

    In head-to-head human evaluations, PinSage recommendations are preferred about 60% of the time.

    “in head-to-head human evaluations our recommendations are preferred about 60% of the time”
    Quote in the paperSubstantial claim
  • Findingown data
    § 1 INTRODUCTION · p. 2

    A/B tests show 30% to 100% improvements in user engagement across various settings.

    “the A/B tests show 30% to 100% improvements in user engagement across various settings.”
    Quote in the paperSubstantial claim
  • Findingown data
    § ABSTRACT · p. 1

    According to offline metrics, user studies, and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives.

    “According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 1 INTRODUCTION · p. 2

    PinSage achieves state-of-the-art performance compared to other scalable deep content-based recommendation algorithms in both item-item recommendation and homefeed recommendation tasks.

    “Through extensive offline metrics, controlled user studies, and A/B tests, we show that our approach achieves state-of-the-art performance compared to other scalable deep content-based recommendation algorithms, in both an item-item recommendation task ( i.e. , related-pin recommendation), as well as a 'homefeed' recommendation task.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 3.2 Model Architecture · p. 4

    Using concatenation operation instead of average operation yields significant performance gains.

    “Empirically we observe significant performance gains when using concatenation operation instead of the average operation as in [21].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] [21] T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR . -

      Not checked in the free run.

  • Classified as probably not a substantial claim. 14
  • May combine two claims. 20
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given
    § 3.1 Problem Setup · p. 3

    The Pinterest graph contains 2 billion pins, 1 billion boards, and over 18 billion edges.

    “Altogether, the Pinterest graph contains 2 billion pins, 1 billion boards, and over 18 billion edges”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given
    § 3.2 Model Architecture · p. 4

    Normalization in Line 3 makes training more stable.

    “the normalization in Line 3 makes training more stable”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Offline Evaluation · p. 8

    PinSage achieves 67% hit-rate and 0.59 MRR.

    “PinSage with our new importance-pooling aggregation and hard negative examples achieves the best performance at 67% hit-rate and 0.59 MRR”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given
    § 3.3 Model Training · p. 6

    Using hard negative items throughout the training procedure doubles the number of epochs needed for training to converge.

    “Using hard negative items throughout the training procedure doubles the number of epochs needed for the training to converge.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Offline Evaluation · p. 8

    PinSage outperforms the top baseline by 40% absolute (150% relative) in hit rate and 22% absolute (60% relative) in MRR.

    “outperforming the top baseline by 40% absolute (150% relative) in terms of the hit rate and also 22% absolute (60% relative) in terms of MRR.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 3.3 Model Training · p. 6

    Hard negative examples are more similar to the query than random negative examples, though not as similar as the positive example.

    “Notice that the hard negative example is significantly more similar to the query, than the random negative example, though not as similar as the positive example.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.1 Experimental Setup · p. 8

    PinSage gives 150% improvement in hit rate and 60% improvement in MRR over the best baseline.

    “Overall, PinSage gives 150% improvement in hit rate and 60% improvement in MRR over the best baseline.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data
    § 4.1 Experimental Setup · p. 7

    Other variants (e.g., based on Kipf et al. [21]) performed significantly worse in development tests and are omitted for brevity.

    “other variants ( e.g. , based on Kipf et al. [21]) performed significantly worse in development tests and are omitted for brevity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] [21] T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR . -

      Not checked in the free run.

  • May combine two claims.

    Findingown data
    § 4.2 Offline Evaluation · p. 8

    The distribution of cosine similarity between random pairs of items demonstrates the effectiveness of PinSage, which has the most spread out distribution.

    “This distribution of cosine similarity between random pairs of items demonstrates the effectiveness of PinSage, which has the most spread out distribution.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.3 User Studies · p. 8

    Visual embeddings generally predict categories and visual similarity well, but occasionally make large mistakes in terms of image semantics.

    “Although visual embeddings generally predict categories and visual similarity well, they occasionally make large mistakes in terms of image semantics.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingreasoning
    § 4.5 Training and Inference Runtime Analysis · p. 9

    The whole inference procedure to generate embeddings for 3 billion items can finish in less than 24 hours.

    “After training completes, due to the highly efficient MapReduce inference pipeline, the whole inference procedure to generate embeddings for 3 billion items can finish in less than 24 hours.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 4.3 User Studies · p. 9

    In head-to-head user studies, PinSage had a higher fraction of wins than the Visual, Annotation, Combined, and Pixie baselines.

    “PinSage vs. Visual | 28.4% | 21.9% | 49.7% | 56.5%”
    • Also stated in: § 4.3 User Studies · p. 9 · plain
      “PinSage vs. Annot. | 36.9% | 14.0% | 49.1% | 72.5%”
    • Also stated in: § 4.3 User Studies · p. 9 · plain
      “PinSage vs. Combined | 22.6% | 15.1% | 57.5% | 60.0%”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.3 User Studies · p. 9

    The proximity of item embeddings corresponds well with the similarity of content, and items of the same category are embedded into the same part of the space.

    “We observe that the proximity of the item embeddings corresponds well with the similarity of content, and that items of the same category are embedded into the same part of the space.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.5 Training and Inference Runtime Analysis · p. 9

    Training on a subgraph containing 300 million items achieved the best performance in terms of hit-rate, reducing runtime by a factor of 6 compared to training on the full graph.

    “Experiments on development data demonstrated that training on a subgraph containing 300 million items could achieve the best performance in terms of hit-rate ( i.e. , further increases in the training set size did not seem to help), reducing the runtime by a factor of 6 compared to training on the full graph.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5 CONCLUSION · p. 10

    PinSage was deployed at Pinterest and comprehensively evaluated on recommendation tasks, with offline metrics, user studies and A/B tests all demonstrating a substantial improvement in recommendation performance.

    “We deployed PinSage at Pinterest and comprehensively evaluated the quality of the learned embeddings on a number of recommendation tasks, with offline metrics, user studies and A/B tests all demonstrating a substantial improvement in recommendation performance.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: A user saving a pin to a board is a high-value action that signifies deep engagement of the user.

  • May combine two claims.

    Findingown data
    § 4.2 Offline Evaluation · p. 8

    The kurtosis of the cosine similarities of PinSage embeddings is 0.43, compared to 2.49 for annotation embeddings and 1.20 for visual embeddings.

    “In particular, the kurtosis of the cosine similarities of PinSage embeddings is 0 . 43, compared to 2 . 49 for annotation embeddings and 1 . 20 for visual embeddings.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.5 Training and Inference Runtime Analysis · p. 9

    There is a diminishing return as neighborhood size T increases, and a two-layer GCN with neighborhood size 50 can best capture the neighborhood information of nodes while still being computationally efficient.

    “We observe a diminishing return as T increases, and find that a two-layer GCN with neighborhood size 50 can best capture the neighborhood information of nodes, while still being computationally efficient.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.3 User Studies · p. 8

    The graph-based Pixie method correctly understands the category of the query but does not find the most relevant items.

    “The graphbased Pixie method, which uses the graph of pin-to-board relations, correctly understands that the category of query is 'plants' and it recommends items in that general category. However, it does not find the most relevant items.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § ABSTRACT · p. 1

    GCN-based methods have set a new standard on countless recommender system benchmarks.

    “Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks.”
    • Also stated in: § Graph Convolutional Neural Networks for Web-Scale Recommender Systems · p. 1 · plain
      “GCN-based methods have set a new standard on countless recommender system benchmarks (see [19] for a survey).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] [19] W. L. Hamilton, R. Ying, and J. Leskovec. 2017. Representation Learning on Graphs: Methods and Applications. IEEE Data Engineering Bulletin (2017). -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 2 RELATED WORK · p. 3

    No previous works have managed to apply GCN algorithms to production-scale data with billions of nodes and edges.

    “no previous works have managed to apply them to production-scale data with billions of nodes and edges”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 3.2 Model Architecture · p. 4

    Normalized embeddings are more efficient for approximate nearest neighbor search.

    “it is more efficient to perform approximate nearest neighbor search for normalized embeddings (Section 3.5).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2 RELATED WORK · p. 3

    GraphSAGE is an inductive variant of GCNs.

    “GraphSAGE is an inductive variant of GCNs”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] [18] W. L. Hamilton, R. Ying, and J. Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NIPS . -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 INTRODUCTION · p. 2

    Traditional GCN algorithms perform graph convolutions by multiplying feature matrices by powers of the full graph Laplacian.

    “Traditional GCN algorithms perform graph convolutions by multiplying feature matrices by powers of the full graph Laplacian.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 1 INTRODUCTION · p. 2

    The producer-consumer architecture ensures maximal GPU utilization during model training.

    “We develop a producer-consumer architecture for constructing minibatches that ensures maximal GPU utilization during model training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 INTRODUCTION · p. 2

    Random sampling is suboptimal for sampling computation graphs.

    “However, random sampling is suboptimal, and we develop a new technique using short random walks to sample the computation graph.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.5 Efficient nearest-neighbor lookups · p. 6

    After the hash function is computed, retrieval of items can be implemented with a two-level retrieval process based on the Weak AND operator.

    “After the hash function is computed, retrieval of items can be implemented with a two-level retrieval process based on the Weak AND operator [5].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] [5] A. Z. Broder, D. Carmel, M. Herscovici, A. Soffer, and J. Zien. 2003. Efficient query evaluation using a two-level retrieval process. In CIKM . - source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 3.4 Node Embeddings via MapReduce · p. 6

    The MapReduce inference pipeline avoids redundant computations and computes each node's latent vector only once.

    “To ensure efficient inference, we develop a MapReduce approach that runs model inference without repeated computations.”
    • Also stated in: § 3.4 Node Embeddings via MapReduce · p. 6 · plain
      “Note that our approach avoids redundant computations and that the latent vector for each node is computed only once.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3.5 Efficient nearest-neighbor lookups · p. 6

    Given PinSage is trained offline and all node embeddings are computed via MapReduce and saved in a database, efficient nearest-neighbor lookup enables the system to serve recommendations online.

    “Given that the PinSage model is trained offline and all node embeddings are computed via MapReduce and saved in a database, the efficient nearest-neighbor lookup operation enables the system to serve recommendations in an online fashion,”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 4.1 Experimental Setup · p. 7

    We do not consider non-deep learning approaches for generating item/content embeddings because other works have already proven state-of-the-art performance of deep learning approaches for generating such embeddings.

    “We also do not consider non-deep learning approaches for generating item/content embeddings, since other works have already proven state-of-the-art performance of deep learning approaches for generating such embeddings [9, 12, 24].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [9] [9] P. Covington, J. Adams, and E. Sargin. 2016. Deep neural networks for youtube recommendations. In RecSys . ACM. - source

      Not checked in the free run.

    • cites [12] [12] A. Van den Oord, S. Dieleman, and B. Schrauwen. 2013. Deep content-based music recommendation. In NIPS . -

      Not checked in the free run.

    • cites [24] [24] F. Monti, M. M. Bronstein, and X. Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks. In NIPS . -

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § 3.5 Efficient nearest-neighbor lookups · p. 6

    PinSage embeddings can be used for downstream recommendation tasks, and in many settings directly used to make recommendations by nearest-neighbor lookups.

    “The embeddings generated by PinSage can be used for a wide range of downstream recommendation tasks, and in many settings we can directly use these embeddings to make recommendations by performing nearest-neighbor lookups in the learned embedding space.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.2 Offline Evaluation · p. 8

    A wide-spread in the embeddings reduces the collision probability of the subsequent LSH algorithm, thus increasing the efficiency of serving the nearest neighbor pins during recommendation.

    “Another important advantage of having such a wide-spread in the embeddings is that it reduces the collision probability of the subsequent LSH algorithm, thus increasing the efficiency of serving the nearest neighbor pins during recommendation.”
    Quote in the paperSubstantial claim
31 of 31

  • The authors state this as an assumption.

    Pins/items u in I are associated with real-valued attributes x_u in R^d.

    “we also assume that the pins/items u ∈ I are associated with real-valued attributes, x u ∈ R d .”

  • The authors state this as an assumption.

    We have access to a set of labeled pairs of items L.

    “we assume that we have access to a set of labeled pairs of items L”

  • The paper relies on this without support.

    Historical user engagement data (a user interacted with pin i immediately after pin q) identifies pairs of related pins suitable as positive training examples.

    “we use historical user engagement data to identify pairs of pins ( q , i ) , where a user interacted with pin i immediately after she interacted with pin q .”

  • The paper relies on this without support.

    Training on a subset of the full graph drastically decreases training time with negligible impact on final performance.

    “Note that training on a subset of the full graph drastically decreased training time, with a negligible impact on final performance.”

  • The paper relies on this without support.

    A user saving a pin to a board is a high-value action that signifies deep engagement of the user.

    “A user saving a pin to a board is a high-value action that signifies deep engagement of the user.”

  • The authors state this as an assumption.

    The input layer-0 nodes are pins/items and the layer-1 nodes are boards/contexts.

    “we assume the input ( i.e. , 'layer-0') nodes are pins/items (and the layer-1 nodes are boards/contexts).”

  • The authors state this as an assumption.

    Only pins (and not boards) have features.

    “since we assume that only pins (and not boards) have features”

  • The paper relies on this without support.

    The visual and annotation embeddings are state-of-the-art deep learning content-based systems currently deployed at Pinterest.

    “The visual and annotation embeddings are state-of-the-art deep learning content-based systems currently deployed at Pinterest to generate representations of pins.”
8 of 8

Cites

Flagged first, then by citations

Research works printed but not linked: 15

  1. J. Bruna W. Zaremba A. Szlam and Y. LeCun. 2014. Spectral networks and locally connected networks on graphs ICLR. J. Bruna W. Zaremba A. Szlam and Y. LeCun. 2014. Spectral networks and locally connected networks on graphs ICLR.
  2. J. Chen T. Ma and C. Xiao. 2018. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. ICLR (2018). J. Chen T. Ma and C. Xiao. 2018. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. ICLR (2018).
  3. H. Dai B. Dai and L. Song. 2016. Discriminative Embeddings of Latent Variable Models for Structured Data ICML. H. Dai B. Dai and L. Song. 2016. Discriminative Embeddings of Latent Variable Models for Structured Data ICML.
  4. M. Defferrard X. Bresson and P. Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering NIPS. M. Defferrard X. Bresson and P. Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering NIPS.
  5. A. Van den Oord S. Dieleman and B. Schrauwen. 2013. Deep content-based music recommendation. In NIPS. A. Van den Oord S. Dieleman and B. Schrauwen. 2013. Deep content-based music recommendation. In NIPS.
  6. D. Duvenaud D. Maclaurin J. Iparraguirre R. Bombarell T. Hirzel A. Aspuru-Guzik and R. P. Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints NIPS. D. Duvenaud D. Maclaurin J. Iparraguirre R. Bombarell T. Hirzel A. Aspuru-Guzik and R. P. Adams. 2015. Convolutional networks on graphs for learning molecular fingerprints NIPS.
  7. Gori M., IEEE International Joint Conference on Neural Networks.
  8. W. L. Hamilton R. Ying and J. Leskovec. 2017. Inductive Representation Learning on Large Graphs. NIPS. W. L. Hamilton R. Ying and J. Leskovec. 2017. Inductive Representation Learning on Large Graphs. NIPS.
  9. Hamilton W. L., 2017, Graphs: Methods and Applications
  10. T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks ICLR. T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks ICLR.
  11. Y. Li D. Tarlow M. Brockschmidt and R. Zemel. 2015. Gated graph sequence neural networks. In ICLR. Y. Li D. Tarlow M. Brockschmidt and R. Zemel. 2015. Gated graph sequence neural networks. In ICLR.
  12. T. Mikolov I Sutskever K. Chen G. S. Corrado and J. Dean. 2013. Distributed representations of words and phrases and their compositionality NIPS. T. Mikolov I Sutskever K. Chen G. S. Corrado and J. Dean. 2013. Distributed representations of words and phrases and their compositionality NIPS.
  13. F. Monti M. M. Bronstein and X. Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks NIPS. F. Monti M. M. Bronstein and X. Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks NIPS.
  14. OpenMP Architecture Review Board. 2015. OpenMP Application Program Interface Version 4.5. (2015). OpenMP Architecture Review Board. 2015. OpenMP Application Program Interface Version 4.5. (2015).
  15. J. You R. Ying X. Ren W. L. Hamilton and J. Leskovec. 2018. GraphRNN: Generating Realistic Graphs using Deep Auto-regressive Models. ICML (2018). J. You R. Ying X. Ren W. L. Hamilton and J. Leskovec. 2018. GraphRNN: Generating Realistic Graphs using Deep Auto-regressive Models. ICML (2018).

Cited by

Sorted by citations

Authors
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J.
Affiliations
  • Stanford University (US)
  • Pinterest (United States) (US)
ORCID
Venue
Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
Publisher
ACM
Published
2018-07-19
Pages
974-983
DOI
10.1145/3219819.3219890
OpenAlex
W2807021761
Semantic Scholar
6c96c2d4a3fbd572fef2d59cb856521ee1746789
Cited by
3,733
References
17
Citations by year
2026: 242 · 2025: 510 · 2024: 579 · 2023: 603 · 2022: 594 · 2021: 624 · 2020: 419 · 2019: 144 · 2018: 15 · 2017: 2
FWCI
235.84
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
Keywords
graph convolutional networks, recommender systems, random walks, scalability, training strategies, Pinterest, large-scale graphs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review