A Comprehensive Survey on Graph Neural Networks

May combine two claims.

Methods in [19] and [20] require eigenvalue decomposition, so their time complexity is O(n^3).

“As methods in [19] and [20] require eigenvalue decomposition, the time complexity is O ( n 3 ) .”

• cites [19] [19] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, 'Spectral networks and locally connected networks on graphs,' in Proc. of ICLR , 2014. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [20] [20] M. Henaff, J. Bruna, and Y. LeCun, 'Deep convolutional networks on graph-structured data,' arXiv preprint arXiv:1506.05163 , 2015. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Gated Graph Neural Network (GGNN) employs a gated recurrent unit (GRU) as a recurrent function, reducing the recurrence to a fixed number of steps.

“Gated Graph Neural Network (GGNN) [17] employs a gated recurrent unit (GRU) [81] as a recurrent function, reducing the recurrence to a fixed number of steps.”

• cites [17] [17] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, 'Gated graph sequence neural networks,' in Proc. of ICLR , 2015. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [81] [81] K. Cho, B. Van Merri¨ enboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, 'Learning phrase representations using rnn encoder-decoder for statistical machine translation,' in Proc. of EMNLP , 2014, pp. 1724-1734. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

SSE updates node hidden states recurrently in a stochastic and asynchronous fashion.

“SSE updates node hidden states recurrently in a stochastic and asynchronous fashion.”

• cites [18] [18] H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song, 'Learning steadystates of iterative algorithms over graphs,' in Proc. of ICML , 2018, pp. 1114-1122. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

SSE alternatively samples a batch of nodes for state update and a batch of nodes for gradient computation.

“It alternatively samples a batch of nodes for state update and a batch of nodes for gradient computation.”

• cites [18] [18] H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song, 'Learning steadystates of iterative algorithms over graphs,' in Proc. of ICML , 2018, pp. 1114-1122. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Stochastic Steady-state Embedding (SSE) proposes a learning algorithm that is more scalable to large graphs.

“Stochastic Steady-state Embedding (SSE) proposes a learning algorithm that is more scalable to large graphs [18].”

• cites [18] [18] H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song, 'Learning steadystates of iterative algorithms over graphs,' in Proc. of ICML , 2018, pp. 1114-1122. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

To maintain stability, the recurrent function of SSE is defined as a weighted average of the historical states and new states.

“To maintain stability, the recurrent function of SSE is defined as a weighted average of the historical states and new states”

• cites [18] [18] H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song, 'Learning steadystates of iterative algorithms over graphs,' in Proc. of ICML , 2018, pp. 1114-1122. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Spectral CNN assumes the filter is a set of learnable parameters and considers graph signals with multiple channels.

“Spectral Convolutional Neural Network (Spectral CNN) [19] assumes the filter g θ = Θ ( k ) i,j is a set of learnable parameters and considers graph signals with multiple channels.”

• cites [19] [19] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, 'Spectral networks and locally connected networks on graphs,' in Proc. of ICLR , 2014. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

AGCN learns hidden structural relations unspecified by the graph adjacency matrix.

“Adaptive Graph Convolutional Network (AGCN) [40] learns hidden structural relations unspecified by the graph adjacency matrix.”

• cites [40] [40] R. Li, S. Wang, F. Zhu, and J. Huang, 'Adaptive graph convolutional neural networks,' in Proc. of AAAI , 2018, pp. 3546-3553. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

GCN assumes K = 1 and lambda_max = 2 to simplify Equation 8.

“Assuming K = 1 and λ max = 2 , Equation 8 is simplified as”

• cites [22] [22] T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

NN4G uses the unnormalized adjacency matrix, which may potentially cause hidden node states to have extremely different scales.

“One difference is that NN4G uses the unnormalized adjacency matrix which may potentially cause hidden node states to have extremely different scales.”

• cites [24] [24] A. Micheli, 'Neural network for graphs: A contextual constructive approach,' IEEE Transactions on Neural Networks , vol. 20, no. 3, pp. 498-511, 2009. - sourceNot checked yet

  Not checked in the free run.

May combine two claims.

As an improvement over Spectral CNN, the filters defined by ChebNet are localized in space, meaning filters can extract local features independently of the graph size.

“As an improvement over Spectral CNN, the filters defined by ChebNet are localized in space, which means filters can extract local features independently of the graph size.”

• cites [21] [21] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proc. of NIPS , 2016, pp. 3844-3852. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

ChebNet approximates the filter by Chebyshev polynomials of the diagonal matrix of eigenvalues.

“Chebyshev Spectral CNN (ChebNet) [21] approximates the filter g θ by Chebyshev polynomials of the diagonal matrix of eigenvalues”

• cites [21] [21] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proc. of NIPS , 2016, pp. 3844-3852. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

CGMM proposes a probabilistic model inspired by NN4G.

“Contextual Graph Markov Model (CGMM) [47] proposes a probabilistic model inspired by NN4G.”

• cites [47] [47] D. Bacciu, F. Errica, and A. Micheli, 'Contextual graph markov model: Adeep and generative approach to graph processing,' in Proc. of ICML , 2018. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

CayleyNet applies Cayley polynomials, which are parametric rational complex functions, to capture narrow frequency bands.

“CayleyNet [23] further applies Cayley polynomials which are parametric rational complex functions to capture narrow frequency bands.”

• cites [23] [23] R. Levie, F. Monti, X. Bresson, and M. M. Bronstein, 'Cayleynets: Graph convolutional neural networks with complex rational spectral filters,' IEEE Transactions on Signal Processing , vol. 67, no. 1, pp. 97-109, 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

DGCN introduces a dual graph convolutional architecture with two graph convolutional layers in parallel.

“Dual Graph Convolutional Network (DGCN) [41] introduces a dual graph convolutional architecture with two graph convolutional layers in parallel.”

• cites [41] [41] C. Zhuang and Q. Ma, 'Dual graph convolutional networks for graphbased semi-supervised classification,' in WWW , 2018, pp. 499-508. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

To restrain the number of parameters and avoid over-fitting, GCN assumes theta = theta0 = -theta1.

“To restrain the number of parameters and avoid over-fitting, GCN further assume θ = θ 0 = -θ 1”

• cites [22] [22] T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

MPNN outlines a general framework of spatial-based ConvGNNs.

“Message Passing Neural Network (MPNN) [27] outlines a general framework of spatial-based ConvGNNs.”

• cites [27] [27] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, 'Neural message passing for quantum chemistry,' in Proc. of ICML , 2017, pp. 1263-1272. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

DCNN regards graph convolutions as a diffusion process.

“Diffusion Convolutional Neural Network (DCNN) [25] regards graph convolutions as a diffusion process.”

• cites [25] [25] J. Atwood and D. Towsley, 'Diffusion-convolutional neural networks,' in Proc. of NIPS , 2016, pp. 1993-2001. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

While maintaining spatial locality, CGMM has the benefit of probabilistic interpretability.

“While maintaining spatial locality, CGMM has the benefit of probabilistic interpretability.”

• cites [47] [47] D. Bacciu, F. Errica, and A. Micheli, 'Contextual graph markov model: Adeep and generative approach to graph processing,' in Proc. of ICML , 2018. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

GraphSage adopts sampling to obtain a fixed number of neighbors for each node.

“GraphSage [42] adopts sampling to obtain a fixed number of neighbors for each node.”

• cites [42] [42] W. Hamilton, Z. Ying, and J. Leskovec, 'Inductive representation learning on large graphs,' in Proc. of NIPS , 2017, pp. 1024-1034. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The calculation of the shortest path adjacency matrix can be expensive with O(n^3) at maximum.

“The calculation of the shortest path adjacency matrix can be expensive with O ( n 3 ) at maximum.”

Probably not a substantial claim: may not pass S3 (load-bearing).

DCNN assumes information is transferred from one node to one of its neighboring nodes with a certain transition probability so that information distribution can reach equilibrium after several rounds.

“It assumes information is transferred from one node to one of its neighboring nodes with a certain transition probability so that information distribution can reach equilibrium after several rounds.”

• cites [25] [25] J. Atwood and D. Towsley, 'Diffusion-convolutional neural networks,' in Proc. of NIPS , 2016, pp. 1993-2001. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

MPNN treats graph convolutions as a message passing process in which information can be passed from one node to another along edges directly.

“It treats graph convolutions as a message passing process in which information can be passed from one node to another along edges directly.”

• cites [27] [27] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, 'Neural message passing for quantum chemistry,' in Proc. of ICML , 2017, pp. 1263-1272. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

MPNN runs K-step message passing iterations to let information propagate further.

“MPNN runs K-step message passing iterations to let information propagate further.”

• cites [27] [27] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, 'Neural message passing for quantum chemistry,' in Proc. of ICML , 2017, pp. 1263-1272. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

DCNN concatenates H(1), H(2), ..., H(K) together as the final model outputs.

“DCNN concatenates H (1) , H (2) , · · · , H ( K ) together as the final model outputs.”

• cites [25] [25] J. Atwood and D. Towsley, 'Diffusion-convolutional neural networks,' in Proc. of NIPS , 2016, pp. 1993-2001. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

PGC partitions a node's neighbors into Q groups based on certain criteria not limited to shortest paths.

“Partition Graph Convolution (PGC) [75] partitions a node's neighbors into Q groups based on certain criteria not limited to shortest paths.”

• cites [75] [75] S. Yan, Y. Xiong, and D. Lin, 'Spatial temporal graph convolutional networks for skeleton-based action recognition,' in Proc. of AAAI , 2018. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In DCNN, the hidden representation matrix H(k) remains the same dimension as the input feature matrix X and is not a function of its previous hidden representation matrix H(k-1).

“Note that in DCNN, the hidden representation matrix H ( k ) remains the same dimension as the input feature matrix X and is not a function of its previous hidden representation matrix H ( k -1) .”

• cites [25] [25] J. Atwood and D. Towsley, 'Diffusion-convolutional neural networks,' in Proc. of NIPS , 2016, pp. 1993-2001. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

DGC sums up outputs at each diffusion step instead of concatenation.

“Diffusion Graph Convolution (DGC) [72] sums up outputs at each diffusion step instead of concatenation.”

• cites [72] [72] Y. Li, R. Yu, C. Shahabi, and Y. Liu, 'Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,' in Proc. of ICLR , 2018. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

GeniePath proposes an LSTM-like gating mechanism to control information flow across graph convolutional layers.

“Apart from applying graph attention spatially, GeniePath [55] further proposes an LSTMlike gating mechanism to control information flow across graph convolutional layers.”

• cites [55] [55] Z. Liu, C. Chen, L. Li, J. Zhou, X. Li, and L. Song, 'Geniepath: Graph neural networks with adaptive receptive paths,' in Proc. of AAAI , 2019. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The ranking criterion of PATCHY-SAN only considers graph structures, which requires heavy computation for data processing.

“The ranking criterion of PATCHY-SAN only considers graph structures, which requires heavy computation for data processing.”

• cites [26] [26] M. Niepert, M. Ahmed, and K. Kutzkov, 'Learning convolutional neural networks for graphs,' in Proc. of ICML , 2016, pp. 2014-2023. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

MoNet introduces node pseudo-coordinates to determine the relative position between a node and its neighbor.

“Mixture Model Network (MoNet) [44] adopts a different approach to assign different weights to a node's neighbors. It introduces node pseudo-coordinates to determine the relative position between a node and its neighbor.”

• cites [44] [44] F. Monti, D. Boscaini, J. Masci, E. Rodola, J. Svoboda, and M. M. Bronstein, 'Geometric deep learning on graphs and manifolds using mixture model cnns,' in Proc. of CVPR , 2017, pp. 5115-5124. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

PATCHY-SAN applies a standard 1D convolutional filter to aggregate neighborhood feature information where the order of the filter's weights corresponds to the order of a node's neighbors.

“PATCHY-SAN applies a standard 1D convolutional filter to aggregate neighborhood feature information where the order of the filter's weights corresponds to the order of a node's neighbors.”

• cites [26] [26] M. Niepert, M. Ahmed, and K. Kutzkov, 'Learning convolutional neural networks for graphs,' in Proc. of ICML , 2016, pp. 2014-2023. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

GraphSage samples a tree rooted at each node by recursively expanding the root node's neighborhood by K steps with a fixed sample size.

“It samples a tree rooted at each node by recursively expanding the root node's neighborhood by K steps with a fixed sample size.”

• cites [42] [42] W. Hamilton, Z. Ying, and J. Leskovec, 'Inductive representation learning on large graphs,' in Proc. of NIPS , 2017, pp. 1024-1034. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

MoNet proposes a Gaussian kernel with learnable parameters to learn the weight function adaptively.

“MoNet additionally proposes a Gaussian kernel with learnable parameters to learn the weight function adaptively.”

• cites [44] [44] F. Monti, D. Boscaini, J. Masci, E. Rodola, J. Svoboda, and M. M. Bronstein, 'Geometric deep learning on graphs and manifolds using mixture model cnns,' in Proc. of CVPR , 2017, pp. 5115-5124. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

LGCN ranks a node's neighbors based on node feature information.

“Largescale Graph Convolutional Network (LGCN) [45] ranks a node's neighbors based on node feature information.”

• cites [45] [45] H. Gao, Z. Wang, and S. Ji, 'Large-scale learnable graph convolutional networks,' in Proc. of KDD . ACM, 2018, pp. 1416-1424. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

While GAT assumes the contributions of attention heads are equal, GAAN introduces a self-attention mechanism which computes an additional attention score for each attention head.

“While GAT assumes the contributions of attention heads are equal, Gated Attention Network (GAAN) [48] introduces a self-attention mechanism which computes an additional attention score for each attention head.”

• cites [48] [48] J. Zhang, X. Shi, J. Xie, H. Ma, I. King, and D.-Y. Yeung, 'Gaan: Gated attention networks for learning on large and spatiotemporal graphs,' in Proc. of UAI , 2018. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

GAT adopts attention mechanisms to learn the relative weights between two connected nodes.

“GAT adopts attention mechanisms to learn the relative weights between two connected nodes.”

• cites [43] [43] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, 'Graph attention networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

PATCHY-SAN orders neighbors of each node according to their graph labelings and selects the top q neighbors.

“PATCHY-SAN [26] orders neighbors of each node according to their graph labelings and selects the top q neighbors.”

• cites [26] [26] M. Niepert, M. Ahmed, and K. Kutzkov, 'Learning convolutional neural networks for graphs,' in Proc. of ICML , 2016, pp. 2014-2023. -Not checked yet

  Not checked in the free run.

May combine two claims.

Cluster-GCN samples a subgraph using a graph clustering algorithm and performs graph convolutions to nodes within the sampled subgraph.

“Cluster-GCN [58] samples a subgraph using a graph clustering algorithm and performs graph convolutions to nodes within the sampled subgraph.”

• cites [58] [58] W.-L. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C.-J. Hsieh, 'Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks,' in Proc. of KDD . ACM, 2019. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

FastGCN interprets graph convolutions as integral transforms of embedding functions of nodes under probability measures.

“It interprets graph convolutions as integral transforms of embedding functions of nodes under probability measures.”

• cites [49] [49] J. Chen, T. Ma, and C. Xiao, 'Fastgcn: fast learning with graph convolutional networks via importance sampling,' in Proc. of ICLR , 2018. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Huang et al. propose an adaptive layer-wise sampling approach where node sampling for the lower layer is conditioned on the top one.

“Huang et al. [51] propose an adaptive layer-wise sampling approach where node sampling for the lower layer is conditioned on the top one.”

• cites [51] [51] W. Huang, T. Zhang, Y. Rong, and J. Huang, 'Adaptive sampling towards fast graph representation learning,' in Proc. of NeurIPS , 2018, pp. 4563-4572. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

This method achieves higher accuracy compared to FastGCN at the cost of employing a much more complicated sampling scheme.

“This method achieves higher accuracy compared to FastGCN at the cost of employing a much more complicated sampling scheme.”

• cites [51] [51] W. Huang, T. Zhang, Y. Rong, and J. Huang, 'Adaptive sampling towards fast graph representation learning,' in Proc. of NeurIPS , 2018, pp. 4563-4572. -Not checked yet

  Not checked in the free run.

• cites [49] [49] J. Chen, T. Ma, and C. Xiao, 'Fastgcn: fast learning with graph convolutional networks via importance sampling,' in Proc. of ICLR , 2018. -Not checked yet

  Not checked in the free run.

May combine two claims.

StoGCN still has to save intermediate states of all nodes, which is memory-consuming for large graphs.

“However, StoGCN still has to save intermediate states of all nodes, which is memory-consuming for large graphs.”

• cites [50] [50] J. Chen, J. Zhu, and L. Song, 'Stochastic training of graph convolutional networks with variance reduction,' in Proc. of ICML , 2018, pp. 941-949. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In Table IV, GCN is the baseline method which conducts full-batch training.

“In Table IV, GCN [22] is the baseline method which conducts the full-batch training.”

• cites [22] [22] T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

May combine two claims.

Earlier graph coarsening algorithms use eigen-decomposition to coarsen graphs based on their topological structure, but these methods suffer from the time complexity issue.

“In some earlier works, the graph coarsening algorithms use eigen-decomposition to coarsen graphs based on their topological structure. However, these methods suffer from the time complexity issue.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Input graphs are first coarsened into multiple levels by the Graclus algorithm; after coarsening, the nodes of the input graph and its coarsened version are rearranged into a balanced binary tree.

“Input graphs are first coarsened into multiple levels by the Graclus algorithm [100]. After coarsening, the nodes of the input graph and its coarsened version are rearranged into a balanced binary tree.”

• cites [21] [21] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proc. of NIPS , 2016, pp. 3844-3852. -Not checked yet

  Not checked in the free run.

• cites [100] [100] I. S. Dhillon, Y. Guan, and B. Kulis, 'Weighted graph cuts without eigenvectors a multilevel approach,' IEEE transactions on pattern analysis and machine intelligence , vol. 29, no. 11, pp. 1944-1957, 2007. -Not checked yet

  Not checked in the free run.

May combine two claims.

Spectral models either need to perform eigenvector computation or handle the whole graph at the same time.

“Spectral models either need to perform eigenvector computation or handle the whole graph at the same time.”

May combine two claims.

Defferrard et al. address this issue by rearranging nodes of a graph in a meaningful way and devise an efficient pooling strategy in ChebNet.

“Defferrard et al. [21] address this issue in another way by rearranging nodes of a graph in a meaningful way. They devise an efficient pooling strategy in their approach ChebNet.”

• cites [21] [21] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proc. of NIPS , 2016, pp. 3844-3852. -Not checked yet

  Not checked in the free run.

May combine two claims.

GraphSage saves memory at the cost of sacrificing time efficiency.

“GraphSage saves memory at the cost of sacrificing time efficiency.”

• cites [42] [42] W. Hamilton, Z. Ying, and J. Leskovec, 'Inductive representation learning on large graphs,' in Proc. of NIPS , 2017, pp. 1024-1034. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Henaff et al. show that performing a simple max/mean pooling at the beginning of the network is especially important to reduce the dimensionality in the graph domain and mitigate the cost of the expensive graph Fourier transform operation.

“Henaff et al. [20] show that performing a simple max/mean pooling at the beginning of the network is especially important to reduce the dimensionality in the graph domain and mitigate the cost of the expensive graph Fourier transform operation.”

• cites [20] [20] M. Henaff, J. Bruna, and Y. LeCun, 'Deep convolutional networks on graph-structured data,' arXiv preprint arXiv:1506.05163 , 2015. - sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The time complexity of StoGCN is the highest, and the bottleneck of the memory remains unsolved.

“The time complexity of StoGCN is the highest, and the bottleneck of the memory remains unsolved.”

• cites [50] [50] J. Chen, J. Zhu, and L. Song, 'Stochastic training of graph convolutional networks with variance reduction,' in Proc. of ICML , 2018, pp. 941-949. -Not checked yet

  Not checked in the free run.

May combine two claims.

The time complexity of Cluster-GCN remains the same as the baseline method since it does not introduce redundant computations.

“The time complexity of Cluster-GCN remains the same as the baseline method since it does not introduce redundant computations.”

• cites [58] [58] W.-L. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C.-J. Hsieh, 'Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks,' in Proc. of KDD . ACM, 2019. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The Graclus algorithm is an alternative to eigen-decomposition to calculate a clustering version of the original graph.

“The Graclus algorithm [100] is an alternative of eigen-decomposition to calculate a clustering version of the original graph.”

• cites [100] [100] I. S. Dhillon, Y. Guan, and B. Kulis, 'Weighted graph cuts without eigenvectors a multilevel approach,' IEEE transactions on pattern analysis and machine intelligence , vol. 29, no. 11, pp. 1944-1957, 2007. -Not checked yet

  Not checked in the free run.

May combine two claims.

Spectral models assume a fixed graph, and any perturbations to a graph would result in a change of eigenbasis.

“They assume a fixed graph. Any perturbations to a graph would result in a change of eigenbasis.”

May combine two claims.

The drawback of DiffPool is that it generates dense graphs after pooling and thereafter the computational complexity becomes O(n^2).

“However, the drawback of DiffPool is that it generates dense graphs after pooling and thereafter the computational complexity becomes O ( n 2 ) .”

• cites [54] [54] Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec, 'Hierarchical graph representation learning with differentiable pooling,' in Proc. of NeurIPS , 2018, pp. 4801-4811. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Pooling such a rearranged signal is much more efficient than pooling the original.

“Pooling such a rearranged signal is much more efficient than pooling the original.”

• cites [21] [21] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proc. of NIPS , 2016, pp. 3844-3852. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Arbitrarily aggregating the balanced binary tree from bottom to top will arrange similar nodes together.

“Arbitrarily aggregating the balanced binary tree from bottom to top will arrange similar nodes together.”

• cites [21] [21] M. Defferrard, X. Bresson, and P. Vandergheynst, 'Convolutional neural networks on graphs with fast localized spectral filtering,' in Proc. of NIPS , 2016, pp. 3844-3852. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The graph's unordered node features from spatial graph convolutions are treated as continuous WL colors, and they are then used to sort nodes.

“The graph's unordered node features from spatial graph convolutions are treated as continuous WL colors [93], and they are then used to sort nodes.”

• cites [52] [52] M. Zhang, Z. Cui, M. Neumann, and Y. Chen, 'An end-to-end deep learning architecture for graph classification,' in Proc. of AAAI , 2018. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [93] [93] B. Weisfeiler and A. Lehman, 'A reduction of a graph to a canonical form and an algebra arising during this reduction,' NauchnoTechnicheskaya Informatsia , vol. 2, no. 9, pp. 12-16, 1968. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The core idea of DiffPool is to learn comprehensive node assignments which consider both topological and feature information of a graph, so Equation 28 can be implemented with any standard ConvGNNs.

“The core idea of this is to learn comprehensive node assignments which consider both topological and feature information of a graph, so Equation 28 can be implemented with any standard ConvGNNs.”

• cites [54] [54] Z. Ying, J. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec, 'Hierarchical graph representation learning with differentiable pooling,' in Proc. of NeurIPS , 2018, pp. 4801-4811. -Not checked yet

  Not checked in the free run.

May combine two claims.

SAGPool considers both node features and graph topology and learns the pooling in a self-attention manner.

“Most recently, the SAGPool [102] approach is proposed, which considers both node features and graph topology and learns the pooling in a self-attention manner.”

• cites [102] [102] J. Lee, I. Lee, and J. Kang, 'Self-attention graph pooling,' in Proc. of ICML , 2019, pp. 3734-3743. -Not checked yet

  Not checked in the free run.

May combine two claims.

Given the number of model parameter p and the number of nodes n, Scarselli et al. derive that the VC dimension of a GNN* is O(p^4 n^2) if it uses the sigmoid or tangent hyperbolic activation and is O(p^2 n) if it uses the piecewise polynomial activation function.

“Given the number of model parameter p and the number of nodes n , Scarselli et al. [103] derive that the VC dimension of a GNN* [15] is O ( p 4 n 2 ) if it uses the sigmoid or tangent hyperbolic activation and is O ( p 2 n ) if it uses the piecewise polynomial activation function.”

• cites [103] [103] F. Scarselli, A. C. Tsoi, and M. Hagenbuchner, 'The vapnikchervonenkis dimension of graph and recursive neural networks,' Neural Networks , vol. 108, pp. 248-259, 2018. - sourceNot checked yet

  Not checked in the free run.

• cites [15] [15] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, 'The graph neural network model,' IEEE Transactions on Neural Networks , vol. 20, no. 1, pp. 61-80, 2009. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Maron et al. theoretically study the characteristics of permutation invariant and equivariant linear layers for graph data.

“Maron et al. [104] theoretically study the characteristics of permutation invariant and equivariant linear layers for graph data.”

• cites [104] [104] H. Maron, H. Ben-Hamu, N. Shamir, and Y. Lipman, 'Invariant and equivariant graph networks,' in ICLR , 2019. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Hammer et al. prove that cascade correlation can approximate functions with structured outputs.

“Hammer et al. [106] prove that cascade correlation can approximate functions with structured outputs.”

• cites [106] [106] B. Hammer, A. Micheli, and A. Sperduti, 'Universal approximation capability of cascade correlation for structures,' Neural Computation , vol. 17, no. 5, pp. 1109-1159, 2005. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The universal approximation capability of GNNs has seldom been studied.

“The universal approximation capability of GNNs has seldom been studied.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Xu et al. show that ConvGNNs under the framework of message passing are not universal approximators of continuous functions defined on multisets.

“Xu et al. [57] show that ConvGNNs under the framework of message passing [27] are not universal approximators of continuous functions defined on multisets.”

• cites [57] [57] K. Xu, W. Hu, J. Leskovec, and S. Jegelka, 'How powerful are graph neural networks,' in Proc. of ICLR , 2019. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [27] [27] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, 'Neural message passing for quantum chemistry,' in Proc. of ICML , 2017, pp. 1263-1272. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

DNGR uses a stacked denoising autoencoder to encode and decode the PPMI matrix via multi-layer perceptrons.

“Deep Neural Network for Graph Representations (DNGR) [59] uses a stacked denoising autoencoder [108] to encode and decode the PPMI matrix via multi-layer perceptrons.”

• cites [59] [59] S. Cao, W. Lu, and Q. Xu, 'Deep neural networks for learning graph representations,' in Proc. of AAAI , 2016, pp. 1145-1152. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [108] [108] P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, 'Extracting and composing robust features with denoising autoencoders,' in Proc. of ICML . ACM, 2008, pp. 1096-1103. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The first loss function enables the learned network embeddings to preserve the node first-order proximity by minimizing the distance between a node's network embedding and its neighbors' network embeddings.

“The first loss function enables the learned network embeddings to preserve the node firstorder proximity by minimizing the distance between a node's network embedding and its neighbors' network embeddings.”

• cites [60] [60] D. Wang, P. Cui, and W. Zhu, 'Structural deep network embedding,' in Proc. of KDD . ACM, 2016, pp. 1225-1234. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

SDNE uses a stacked autoencoder to preserve the node first-order proximity and second-order proximity jointly.

“Concurrently, Structural Deep Network Embedding (SDNE) [60] uses a stacked autoencoder to preserve the node first-order proximity and second-order proximity jointly.”

• cites [60] [60] D. Wang, P. Cui, and W. Zhu, 'Structural deep network embedding,' in Proc. of KDD . ACM, 2016, pp. 1225-1234. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

SDNE proposes two loss functions on the outputs of the encoder and the outputs of the decoder separately.

“SDNE proposes two loss functions on the outputs of the encoder and the outputs of the decoder separately.”

• cites [60] [60] D. Wang, P. Cui, and W. Zhu, 'Structural deep network embedding,' in Proc. of KDD . ACM, 2016, pp. 1225-1234. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GAEs learn network embeddings using an encoder to extract network embeddings and using a decoder to enforce network embeddings to preserve the graph topological information such as the PPMI matrix and the adjacency matrix.

“GAEs learn network embeddings using an encoder to extract network embeddings and using a decoder to enforce network embeddings to preserve the graph topological information such as the PPMI matrix and the adjacency matrix.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Early studies on graph neural networks fall into the category of recurrent graph neural networks (RecGNNs).

“These early studies fall into the category of recurrent graph neural networks (RecGNNs).”

• cites [14] [14] M. Gori, G. Monfardini, and F. Scarselli, 'A new model for learning in graph domains,' in Proc. of IJCNN , vol. 2. IEEE, 2005, pp. 729-734. - sourceNot checked yet

  Not checked in the free run.

• cites [15] [15] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, 'The graph neural network model,' IEEE Transactions on Neural Networks , vol. 20, no. 1, pp. 61-80, 2009. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [16] [16] C. Gallicchio and A. Micheli, 'Graph echo state networks,' in IJCNN . IEEE, 2010, pp. 1-8. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

The complexity of graph data has imposed significant challenges on existing machine learning algorithms.

“The complexity of graph data has imposed significant challenges on existing machine learning algorithms.”

• Also stated in: § I. INTRODUCTION · p. 1 · plain

  “The complexity of graph data has imposed significant challenges on existing machine learning algorithms.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A graph convolution can be generalized from a 2D convolution.

“For example, a graph convolution can be generalized from a 2D convolution.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The notion of graph neural networks was initially outlined in Gori et al. (2005) and further elaborated in Scarselli et al. (2009) and Gallicchio et al. (2010).

“The notion of graph neural networks was initially outlined in Gori et al. (2005) [14] and further elaborated in Scarselli et al. (2009) [15], and Gallicchio et al. (2010) [16].”

• cites [14] [14] M. Gori, G. Monfardini, and F. Scarselli, 'A new model for learning in graph domains,' in Proc. of IJCNN , vol. 2. IEEE, 2005, pp. 729-734. - sourceNot checked yet

  Not checked in the free run.

• cites [15] [15] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, 'The graph neural network model,' IEEE Transactions on Neural Networks , vol. 20, no. 1, pp. 61-80, 2009. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [16] [16] C. Gallicchio and A. Micheli, 'Graph echo state networks,' in IJCNN . IEEE, 2010, pp. 1-8. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

A core assumption of existing machine learning algorithms is that instances are independent of each other.

“Furthermore, a core assumption of existing machine learning algorithms is that instances are independent of each other.”

May combine two claims.

Hamilton et al. cover a limited number of GNNs with a focus on addressing the problem of network embedding.

“Hamilton et al. [10] cover a limited number of GNNs with a focus on addressing the problem of network embedding.”

• cites [10] [10] W. L. Hamilton, R. Ying, and J. Leskovec, 'Representation learning on graphs: Methods and applications,' in Proc. of NIPS , 2017, pp. 1024-1034. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Existing surveys on graph neural networks only include some of the GNNs and examine a limited number of works, thereby missing the most recent development of GNNs.

“In summary, existing surveys only include some of the GNNs and examine a limited number of works, thereby missing the most recent development of GNNs.”

• cites [9] [9] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, 'Geometric deep learning: going beyond euclidean data,' IEEE Signal Processing Magazine , vol. 34, no. 4, pp. 18-42, 2017. - sourceNot checked yet

  Not checked in the free run.

• cites [10] [10] W. L. Hamilton, R. Ying, and J. Leskovec, 'Representation learning on graphs: Methods and applications,' in Proc. of NIPS , 2017, pp. 1024-1034. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [11] [11] P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner et al. , 'Relational inductive biases, deep learning, and graph networks,' arXiv preprint arXiv:1806.01261 , 2018. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Graphs can be irregular: a graph may have a variable size of unordered nodes, and nodes may have different numbers of neighbors, which makes operations such as convolutions easy in the image domain but difficult to apply to the graph domain.

“As graphs can be irregular, a graph may have a variable size of unordered nodes, and nodes from a graph may have a different number of neighbors, resulting in some important operations (e.g., convolutions) being easy to compute in the image domain, but difficult to apply to the graph domain.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Sperduti et al. (1997) first applied neural networks to directed acyclic graphs, which motivated early studies on GNNs.

“Sperduti et al. (1997) [13] first applied neural networks to directed acyclic graphs, which motivated early studies on GNNs.”

• cites [13] [13] A. Sperduti and A. Starita, 'Supervised neural networks for the classification of structures,' IEEE Transactions on Neural Networks , vol. 8, no. 3, pp. 714-735, 1997. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Battaglia et al. position graph networks as the building blocks for learning from relational data, reviewing part of GNNs under a unified framework.

“Battaglia et al. [11] position graph networks as the building blocks for learning from relational data, reviewing part of GNNs under a unified framework.”

• cites [11] [11] P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner et al. , 'Relational inductive biases, deep learning, and graph networks,' arXiv preprint arXiv:1806.01261 , 2018. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

RecGNNs learn a target node's representation by propagating neighbor information in an iterative manner until a stable fixed point is reached.

“They learn a target node's representation by propagating neighbor information in an iterative manner until a stable fixed point is reached.”

• cites [14] [14] M. Gori, G. Monfardini, and F. Scarselli, 'A new model for learning in graph domains,' in Proc. of IJCNN , vol. 2. IEEE, 2005, pp. 729-734. - sourceNot checked yet

  Not checked in the free run.

• cites [15] [15] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, 'The graph neural network model,' IEEE Transactions on Neural Networks , vol. 20, no. 1, pp. 61-80, 2009. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [16] [16] C. Gallicchio and A. Micheli, 'Graph echo state networks,' in IJCNN . IEEE, 2010, pp. 1-8. - sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Bronstein et al. give an overview of deep learning methods in the non-Euclidean domain, including graphs and manifolds, but this survey mainly reviews convolutional GNNs.

“Using the term geometric deep learning , Bronstein et al. [9] give an overview of deep learning methods in the non-Euclidean domain, including graphs and manifolds. Although it is the first review on GNNs, this survey mainly reviews convolutional GNNs.”

• cites [9] [9] M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, 'Geometric deep learning: going beyond euclidean data,' IEEE Signal Processing Magazine , vol. 34, no. 4, pp. 18-42, 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

An image can be considered as a special case of graphs where pixels are connected by adjacent pixels.

“As illustrated in Figure 1, an image can be considered as a special case of graphs where pixels are connected by adjacent pixels.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For graph generation, some methods generate nodes and edges of a graph step by step while other methods output a graph all at once.

“For graph generation, some methods generate nodes and edges of a graph step by step while other methods output a graph all at once.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The process of propagating neighbor information iteratively until a stable fixed point is computationally expensive.

“This process is computationally expensive, and recently there have been increasing efforts to overcome these challenges [17], [18].”

• cites [17] [17] Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel, 'Gated graph sequence neural networks,' in Proc. of ICLR , 2015. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [18] [18] H. Dai, Z. Kozareva, B. Dai, A. Smola, and L. Song, 'Learning steadystates of iterative algorithms over graphs,' in Proc. of ICML , 2018, pp. 1114-1122. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The main distinction between GNNs and network embedding is that GNNs are a group of neural network models designed for various tasks while network embedding covers various kinds of methods targeting the same task.

“The main distinction between GNNs and network embedding is that GNNs are a group of neural network models which are designed for various tasks while network embedding covers various kinds of methods targeting the same task.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The first prominent research on spectral-based ConvGNNs was presented by Bruna et al. (2013), which developed a graph convolution based on spectral graph theory.

“The first prominent research on spectral-based ConvGNNs was presented by Bruna et al. (2013) [19], which developed a graph convolution based on the spectral graph theory.”

• cites [19] [19] J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, 'Spectral networks and locally connected networks on graphs,' in Proc. of ICLR , 2014. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Graph kernels can embed graphs or nodes into vector spaces by a mapping function, but this mapping function is deterministic rather than learnable.

“Similar to GNNs, graph kernels can embed graphs or nodes into vector spaces by a mapping function. The difference is that this mapping function is deterministic rather than learnable.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The idea of message passing is inherited by spatial-based convolutional graph neural networks from RecGNNs.

“In particular, the idea of message passing is inherited by spatialbased convolutional graph neural networks.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Many current STGNN approaches integrate graph convolutions to capture spatial dependency with RNNs or CNNs to model temporal dependency.

“Many current approaches integrate graph convolutions to capture spatial dependency with RNNs or CNNs to model the temporal dependency.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The research of spatial-based ConvGNNs started much earlier than spectral-based ConvGNNs.

“The research of spatial-based ConvGNNs started much earlier than spectralbased ConvGNNs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Different from RecGNNs, ConvGNNs stack multiple graph convolutional layers to extract high-level node representations.

“Different from RecGNNs, ConvGNNs stack multiple graph convolutional layers to extract high-level node representations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In 2009, Micheli et al. first addressed graph mutual dependency by architecturally composite nonrecursive layers while inheriting ideas of message passing from RecGNNs.

“In 2009, Micheli et al. [24] first addressed graph mutual dependency by architecturally composite nonrecursive layers while inheriting ideas of message passing from RecGNNs.”

• cites [24] [24] A. Micheli, 'Neural network for graphs: A contextual constructive approach,' IEEE Transactions on Neural Networks , vol. 20, no. 3, pp. 498-511, 2009. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Graph kernels are historically dominant techniques to solve the problem of graph classification.

“Graph kernels are historically dominant techniques to solve the problem of graph classification [36], [37], [38].”

• cites [36] [36] S. V. N. Vishwanathan, N. N. Schraudolph, R. Kondor, and K. M. Borgwardt, 'Graph kernels,' Journal of Machine Learning Research , vol. 11, no. Apr, pp. 1201-1242, 2010. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [37] [37] N. Shervashidze, P. Schweitzer, E. J. v. Leeuwen, K. Mehlhorn, and K. M. Borgwardt, 'Weisfeiler-lehman graph kernels,' Journal of Machine Learning Research , vol. 12, no. Sep, pp. 2539-2561, 2011. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [38] [38] N. Navarin and A. Sperduti, 'Approximated neighbours minhash graph node kernel.' in Proc. of ESANN , 2017. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

ConvGNNs play a central role in building up many other complex GNN models.

“ConvGNNs play a central role in building up many other complex GNN models.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Network embedding contains non-deep learning methods such as matrix factorization and random walks.

“On the other hand, network embedding contains other non-deep learning methods such as matrix factorization [33], [34] and random walks [35].”

• cites [33] [33] X. Shen, S. Pan, W. Liu, Y.-S. Ong, and Q.-S. Sun, 'Discrete network embedding,' in Proc. of IJCAI , 2018, pp. 3549-3555. - sourceNot checked yet

  Not checked in the free run.

• cites [34] [34] H. Yang, S. Pan, P. Zhang, L. Chen, D. Lian, and C. Zhang, 'Binarized attributed network embedding,' in Proc. of ICDM . IEEE, 2018. - sourceNot checked yet

  Not checked in the free run.

• cites [35] [35] B. Perozzi, R. Al-Rfou, and S. Skiena, 'Deepwalk: Online learning of social representations,' in Proc. of KDD . ACM, 2014, pp. 701-710. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GAEs are used to learn network embeddings and graph generative distributions.

“GAEs are used to learn network embeddings and graph generative distributions.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

RecGNNs are conceptually important and inspired later research on convolutional graph neural networks.

“RecGNNs are conceptually important and inspired later research on convolutional graph neural networks.”

Probably not a substantial claim: may not pass S3 (load-bearing).

GNNs can address the network embedding problem through a graph autoencoder framework.

“Therefore, GNNs can address the network embedding problem through a graph autoencoder framework.”

May combine two claims.

GNNs directly perform graph classification based on extracted graph representations and therefore are much more efficient than graph kernel methods.

“GNNs, on one hand, directly perform graph classification based on the extracted graph representations and therefore are much more efficient than graph kernel methods.”

Probably not a substantial claim: may not pass S3 (load-bearing).

GraphESN extends echo state networks to improve the training efficiency of GNN*.

“In follow-up works, Graph Echo State Network (GraphESN) [16] extends echo state networks to improve the training efficiency of GNN*.”

• cites [16] [16] C. Gallicchio and A. Micheli, 'Graph echo state networks,' in IJCNN . IEEE, 2010, pp. 1-8. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GraphESN consists of an encoder and an output layer; the encoder is randomly initialized and requires no training, and it implements a contractive state transition function to recurrently update node states until the global graph state reaches convergence.

“GraphESN consists of an encoder and an output layer. The encoder is randomly initialized and requires no training. It implements a contractive state transition function to recurrently update node states until the global graph state reaches convergence.”

• cites [16] [16] C. Gallicchio and A. Micheli, 'Graph echo state networks,' in IJCNN . IEEE, 2010, pp. 1-8. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The time complexity of several methods is missing in Table III because these methods either lack a time complexity analysis in their papers or report the time complexity of their overall models or algorithms.

“The time complexity of several methods are missing in Table III. These methods either lack a time complexity analysis in their papers or report the time complexity of their overall models or algorithms.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

To ensure convergence, the recurrent function f(·) must be a contraction mapping, which shrinks the distance between two points after projecting them into a latent space.

“To ensure convergence, the recurrent function f ( · ) must be a contraction mapping, which shrinks the distance between two points after projecting them into a latent space.”

May combine two claims.

GNN* alternates the stage of node state propagation and the stage of parameter gradient computation to minimize a training objective; this strategy enables GNN* to handle cyclic graphs.

“GNN* alternates the stage of node state propagation and the stage of parameter gradient computation to minimize a training objective. This strategy enables GNN* to handle cyclic graphs.”

May combine two claims.

In these methods, the computation of each node v_i's representation involves its d_i neighbors, and the sum of d_i over all nodes exactly equals the number of edges.

“This is because in these methods the computation of each node v i 's representation involves its d i neighbors, and the sum of d i over all nodes exactly equals the number of edges.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In the case of f(·) being a neural network, a penalty term has to be imposed on the Jacobian matrix of parameters.

“In the case of f ( · ) being a neural network, a penalty term has to be imposed on the Jacobian matrix of parameters.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Constrained by computational power, earlier research mainly focused on directed acyclic graphs.

“Constrained by computational power, earlier research mainly focused on directed acyclic graphs [13], [80].”

• cites [13] [13] A. Sperduti and A. Starita, 'Supervised neural networks for the classification of structures,' IEEE Transactions on Neural Networks , vol. 8, no. 3, pp. 714-735, 1997. - sourceNot checked yet

  Not checked in the free run.

• cites [80] [80] A. Micheli, D. Sona, and A. Sperduti, 'Contextual processing of structured data by recursive cascade correlation,' IEEE Transactions on Neural Networks , vol. 15, no. 6, pp. 1396-1410, 2004. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Spectral-based approaches define graph convolutions by introducing filters from the perspective of graph signal processing, where the graph convolutional operation is interpreted as removing noises from graph signals.

“Spectralbased approaches define graph convolutions by introducing filters from the perspective of graph signal processing [82] where the graph convolutional operation is interpreted as removing noises from graph signals.”

• cites [82] [82] D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, 'The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,' IEEE Signal Processing Magazine , vol. 30, no. 3, pp. 83-98, 2013. - sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

ConvGNNs fall into two categories, spectral-based and spatial-based.

“ConvGNNs fall into two categories, spectral-based and spatial-based.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Spatial-based approaches inherit ideas from RecGNNs to define graph convolutions by information propagation.

“Spatial-based approaches inherit ideas from RecGNNs to define graph convolutions by information propagation.”

May combine two claims.

Since GCN bridged the gap between spectral-based and spatial-based approaches, spatial-based methods have developed rapidly recently due to their attractive efficiency, flexibility, and generality.

“Since GCN [22] bridged the gap between spectral-based approaches and spatial-based approaches, spatial-based methods have developed rapidly recently due to its attractive efficiency, flexibility, and generality.”

• cites [22] [22] T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Spectral-based ConvGNNs all follow the spectral graph convolution definition, and the key difference lies in the choice of the filter g_theta.

“Spectral-based ConvGNNs all follow this definition. The key difference lies in the choice of the filter g θ .”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Graph convolutions are more efficient and convenient to composite with other neural networks, so the popularity of ConvGNNs has been rapidly growing in recent years.

“As graph convolutions are more efficient and convenient to composite with other neural networks, the popularity of ConvGNNs has been rapidly growing in recent years.”

Probably not a substantial claim: may not pass S3 (load-bearing).

From a spatial-based perspective, GCN can be considered as aggregating feature information from a node's neighborhood.

“From a spatial-based perspective, GCN can be considered as aggregating feature information from a node's neighborhood.”

• cites [22] [22] T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Several recent works made incremental improvements over GCN by exploring alternative symmetric matrices.

“Several recent works made incremental improvements over GCN [22] by exploring alternative symmetric matrices.”

• cites [22] [22] T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in Proc. of ICLR , 2017. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Spatial-based ConvGNNs share the same idea of information propagation/message passing with RecGNNs.

“From another perspective, spatial-based ConvGNNs share the same idea of information propagation/message passing with RecGNNs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Spatial-based methods define graph convolutions based on a node's spatial relations.

“Analogous to the convolutional operation of a conventional CNN on an image, spatial-based methods define graph convolutions based on a node's spatial relations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The spatial graph convolutional operation essentially propagates node information along edges.

“The spatial graph convolutional operation essentially propagates node information along edges.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The aggregation function should be invariant to the permutations of node orderings such as a mean, sum or max function.

“The aggregation function should be invariant to the permutations of node orderings such as a mean, sum or max function.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Using the power of a transition probability matrix implies that distant neighbors contribute very little information to a central node.

“Using the power of a transition probability matrix implies that distant neighbors contribute very little information to a central node.”

May combine two claims.

As the neighborhood search is restricted within the sampled subgraph, Cluster-GCN is capable of handling larger graphs and using deeper architectures at the same time, in less time and with less memory.

“As the neighborhood search is also restricted within the sampled subgraph, Cluster-GCN is capable of handling larger graphs and using deeper architectures at the same time, in less time and with less memory.”

• cites [58] [58] W.-L. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C.-J. Hsieh, 'Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks,' in Proc. of KDD . ACM, 2019. -Not checked yet

  Not checked in the free run.

May combine two claims.

Spatial-based models perform graph convolutions locally on each node where weights can be easily shared across different locations and structures.

“Spatialbased models, on the other hand, perform graph convolutions locally on each node where weights can be easily shared across different locations and structures.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Spectral models have a theoretical foundation in graph signal processing.

“Spectral models have a theoretical foundation in graph signal processing.”

May combine two claims.

Even with attention mechanisms, the reduction operation such as sum pooling is not satisfactory since it makes the embedding inefficient: a fixed-size embedding is generated regardless of the graph size.

“Even with attention mechanisms, the reduction operation (such as sum pooling) is not satisfactory since it makes the embedding inefficient: a fixed-size embedding is generated regardless of the graph size.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Mean/max/sum pooling is the most primitive and effective way to implement down-sampling since calculating the mean/max/sum value in the pooling window is fast.

“Nowadays, mean/max/sum pooling is the most primitive and effective way to implement down-sampling since calculating the mean/max/sum value in the pooling window is fast:”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

After a GNN generates node features, using all these features directly can be computationally challenging, so a down-sampling strategy is needed.

“After a GNN generates node features, we can use them for the final task. But using all these features directly can be computationally challenging, thus, a down-sampling strategy is needed.”