CoChat

Co-embedding of edges and nodes with deep graph convolutional neural networks

Open access

The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.

Abstract

Graph neural networks (GNNs) have significant advantages in dealing with non-Euclidean data and have been widely used in various fields. However, most of the existing GNN models face two main challenges: (1) Most GNN models built upon the message-passing framework exhibit a shallow structure, which hampers their ability to efficiently transmit information between distant nodes. To address this, we aim to propose a novel message-passing framework, enabling the construction of GNN models with deep architectures akin to convolutional neural networks (CNNs), potentially comprising dozens or even hundreds of layers. (2) Existing models often approach the learning of edge and node features as separate tasks. To overcome this limitation, we aspire to develop a deep graph convolutional neural network learning framework capable of simultaneously acquiring edge embeddings and node embeddings. By utilizing the learned multi-dimensional edge feature matrix, we construct multi-channel filters to more effectively capture accurate node features. To address these challenges, we propose the Co-embedding of Edges and Nodes with Deep Graph Convolutional Neural Networks (CEN-DGCNN). In our approach, we propose a novel message-passing framework that can fully integrate and utilize both node features and multi-dimensional edge features. Based on this framework, we develop a deep graph convolutional neural network model that prevents over-smoothing and obtains node non-local structural features and refined high-order node features by extracting long-distance dependencies between nodes and utilizing multi-dimensional edge features. Moreover, we propose a novel graph convolutional layer that can learn node embeddings and multi-dimensional edge embeddings simultaneously. The layer updates multi-dimensional edge embeddings across layers based on node features and an attention mechanism, which enables efficient utilization and fusion of both node and edge features. Additionally, we propose a multi-dimensional edge feature encoding method based on directed edges, and use the resulting multi-dimensional edge feature matrix to construct a multi-channel filter to filter the node information. Lastly, extensive experiments show that CEN-DGCNN outperforms a large number of graph neural network baseline methods, demonstrating the effectiveness of our proposed method.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-023-44224-1

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 (a) The edge feature representation and node feature representation in ordinary GCN; (b) The edge feature representation and node feature representation used in our proposed CEN-DGCNN.
Figure 1 (a) The edge feature representation and node feature representation in ordinary GCN; (b) The edge feature representation and node feature representation used in our proposed CEN-DGCNN.CC BY · PubMed Central
Figure 2 The overall architecture of Co-embedding of Edges and Nodes with Deep Graph Convolutional Neural Network (CEN-DGCNN).
Figure 2 The overall architecture of Co-embedding of Edges and Nodes with Deep Graph Convolutional Neural Network (CEN-DGCNN).CC BY · PubMed Central
Figure 3 The novel graph neural network message passing framework proposed by us.
Figure 3 The novel graph neural network message passing framework proposed by us.CC BY · PubMed Central
Figure 4 Edge features update and node features update of CEN-DGCNN.
Figure 4 Edge features update and node features update of CEN-DGCNN.CC BY · PubMed Central
Figure 5 MAD values of different layers of CEN-DGCNN on 5 datasets.
Figure 5 MAD values of different layers of CEN-DGCNN on 5 datasets.CC BY · PubMed Central
Figure 6 MAD values of different layers of regular GCN on 5 datasets.
Figure 6 MAD values of different layers of regular GCN on 5 datasets.CC BY · PubMed Central
Figure 7 t-SNE Visualization of node representations learned by CEN-DGCNN.
Figure 7 t-SNE Visualization of node representations learned by CEN-DGCNN.CC BY · PubMed Central
Figure 8 Node feature visualization results of CEN-DGCNN with various edge feature encoding methods in the Cora dataset.
Figure 8 Node feature visualization results of CEN-DGCNN with various edge feature encoding methods in the Cora dataset.CC BY · PubMed Central
Figure 9 Node feature visualization results of CEN-DGCNN with different message passing frameworks and edge feature construction methods in the Citeseer dataset.
Figure 9 Node feature visualization results of CEN-DGCNN with different message passing frameworks and edge feature construction methods in the Citeseer dataset.CC BY · PubMed Central
Figure 10 CEN-DGCNN Model Analysis. (a) The running time of the CEN-DGCNN model under different-dimensional edge feature constructions. (b) Attention weight distribution on the Citeseer dataset. (c) The performance of CEN-DGCNN is influenced by the value of the hyperparameter λ.
Figure 10 CEN-DGCNN Model Analysis. (a) The running time of the CEN-DGCNN model under different-dimensional edge feature constructions. (b) Attention weight distribution on the Citeseer dataset. (c) The performance of CEN-DGCNN is influenced by the value of the hyperparameter λ.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 39 references

Research works printed but not linked: 26

  1. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning (eds. Precup, D. & Teh, Y. W.) vol. 70 1263–1272 (PMLR, 2017).
  2. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. CoRR abs/1609.0 (2016).
  3. Veličković, P. et al. Graph attention networks. In 6th International Conference on Learning Representations, {ICLR} 2018, Vancouver, BC, Canada, April 30–May 3, 2018, Conference Track Proceedings (OpenReview.net, 2018).
  4. Li, Q., Han, Z. & Wu, X.-M. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the Thirty-Second {AAAI} Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th {AAAI} Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New (eds. McIlraith, S. A. & Weinberger, K. Q.) 3538–3545 ({AAAI} Press, 2018).
  5. Rusch, T. K., Chamberlain, B., Rowbottom, J., Mishra, S. & Bronstein, M. M. Graph-coupled oscillator networks. In International Conference on Machine Learning, {ICML} 2022, 17–23 July 2022, Baltimore, Maryland, {USA} (eds. Chaudhuri, K. et al.) vol. 162, 18888–18909 (PMLR, 2022).
  6. Rusch, T. K., Chamberlain, B. P., Mahoney, M. W., Bronstein, M. M. & Mishra, S. Gradient gating for deep multi-rate learning on graphs. In The Eleventh International Conference on Learning Representations, {ICLR} 2023, Kigali, Rwanda, May 1–5, 2023 (OpenReview.net, 2023).
  7. Bodnar, C., Giovanni, F. Di, Chamberlain, B. P., Lió, P. & Bronstein, M. M. Neural sheaf diffusion: {A} Topological perspective on heterophily and oversmoothing in GNNs. in NeurIPS (2022).
  8. Giovanni, F., Di Rowbottom, J. & Chamberlain, B. P., Markovich, T. & Bronstein, M. M. Graph neural networks as gradient flows. CoRR abs/2206.1 (2022).
  9. Oono, K. & Suzuki, T. Graph neural networks exponentially lose expressive power for node classification. In 8th International Conference on Learning Representations, {ICLR} 2020, Addis Ababa, Ethiopia, April 26–30, 2020 (OpenReview.net, 2020).
  10. Alon, U. & Yahav, E. On the Bottleneck of graph neural networks and its practical implications. In 9th International Conference on Learning Representations, {ICLR} 2021, Virtual Event, Austria, May 3–7, 2021 (OpenReview.net, 2021).
  11. Chen, M., Wei, Z., Huang, Z., Ding, B. & Li, Y. Simple and deep graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning, {ICML} 2020, 13–18 July 2020, Virtual Event vol. 119, 1725–1735 (PMLR, 2020).
  12. Zhao, L. & Akoglu, L., PairNorm: Tackling oversmoothing in GNNs. In 8th International Conference on Learning Representations, {ICLR} 2020, Addis Ababa, Ethiopia, April 26–30, 2020 (OpenReview.net, 2020).
  13. Li, G., Xiong, C., Thabet, A. K. & Ghanem, B. DeeperGCN: All you need to train deeper GCNs. CoRR abs/2006.0 (2020).
  14. Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).
  15. Rong, Y., Huang, W., Xu, T. & Huang, J. DropEdge: Towards deep graph convolutional networks on node classification. In 8th International Conference on Learning Representations, {ICLR} 2020, Addis Ababa, Ethiopia, April 26–30, 2020 (OpenReview.net, 2020).
  16. Huang, W., Rong, Y., Xu, T., Sun, F. & Huang, J. Tackling over-smoothing for general graph convolutional networks. CoRR abs/2008.0, (2020).
  17. Corso, G., Cavalleri, L., Beaini, D., Liò, P. & Veličković, P. Principal neighbourhood aggregation for graph nets. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, virtual (eds. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.-F. & Lin, H.-T.) (2020).
  18. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, {ICLR} 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (eds. Bengio, Y. & LeCun, Y.) (2015).
  19. Yang, Y. & Li, D. {NENN:} Incorporate node and edge features in graph neural networks. In Proceedings of the 12th Asian Conference on Machine Learning, {ACML} 2020, 18–20 November 2020, Bangkok, Thailand (eds. Pan, S. J. & Sugiyama, M.) vol. 129 593–608 (PMLR, 2020).
  20. Klicpera, J., Bojchevski, A. & Günnemann, S. Predict then propagate: Graph neural networks meet personalized pagerank. In 7th International Conference on Learning Representations, {ICLR} 2019, New Orleans, LA, USA, May 6–9, 2019 (OpenReview.net, 2019).
  21. Kingma, D. P. & Ba, J. Adam: {A} method for stochastic optimization. In 3rd International Conference on Learning Representations, {ICLR} 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (eds. Bengio, Y. & LeCun, Y.) (2015).
  22. Klicpera, J., Weißenberger, S. & Günnemann, S. Diffusion improves graph learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8–14, 2019, Vancouver, BC, Canada (eds. Wallach, H. M. et al.) 13333–13345 (2019).
  23. Kipf, T. N. & Welling, M. Variational graph auto-encoders. CoRR abs/1611.0 (2016).
  24. Hamilton, W., Ying, Z. & Leskovec, J. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems (eds. Guyon, I. et al.) vol. 30 (Curran Associates, Inc., 2017).
  25. Li, G., Müller, M., Ghanem, B. & Koltun, V. Training graph neural networks with 1000 layers. In Proceedings of the 38th International Conference on Machine Learning, {ICML} 2021, 18–24 July 2021, Virtual Event (eds. Meila, M. & Zhang, T.) vol. 139, 6437–6449 (PMLR, 2021).
  26. Chen, J., Ma, T. & Xiao, C. FastGCN: Fast learning with graph convolutional networks via importance sampling. In 6th International Conference on Learning Representations, {ICLR} 2018, Vancouver, BC, Canada, April 30–May 3, 2018, Conference Track Proceedings (OpenReview.net, 2018).

Cited by

See all 46

Metadata

Authors
Zhou, Y., Huo, H., Hou, Z., Bu, L., Mao, J., Wang, Y., Lv, X., & Bu, F.
Affiliations
  • China People's Public Security University (CN)
  • China Academy of Railway Sciences (CN)
Corresponding author
Bu, Fanliang
Venue
Scientific Reports
Published
2023-10-08
Volume
13
Issue
1
Pages
16966-16966
ISSN
2045-2322 (ISSN-L)
OpenAlex
W4387439537
Cited by
33
References
36
Citations by year
2026: 11 · 2025: 13 · 2024: 9
FWCI
3.11
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Natural Science Foundation of China-China State Railway Group Co., Ltd. Railway Basic Research Joint Fund (No.U2268217)
  • Scientific Funding for China Academy of Railway Sciences Corporation Limited (No.2021YJ183)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Graph Theory and Algorithms
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
Keywords
graph neural networks, edge embeddings, node embeddings, co-embedding, message passing, attention mechanism, over-smoothing, long-distance dependencies, multichannel filters, directed edges

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Co-embedding of edges and nodes with deep graph convolutional neural networks. https://app.cochat.ai/papers/10.1038/s41598-023-44224-1
The paper
Zhou, Y., Huo, H., Hou, Z., Bu, L., Mao, J., Wang, Y., Lv, X., & Bu, F. (2023). Co-embedding of edges and nodes with deep graph convolutional neural networks. Scientific Reports. https://doi.org/10.1038/s41598-023-44224-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.