Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 389

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.

Self-Supervised Learning of Graph Neural Networks: A Unified Review

Open access
Xie, Yaochen, Xu, Zhao, Zhang, Jingtun, Wang, Zhengyang, Ji, Shuiwang · 2023 · IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Deep models trained in supervised mode have achieved remarkable success on a variety of tasks. When labeled samples are limited, self-supervised learning (SSL) is emerging as a new paradigm for making use of large amounts of unlabeled samples. SSL has achieved promising performance on natural language and image learning tasks. Recently, there is a trend to extend such success to graph data using graph neural networks (GNNs). In this survey, we provide a unified review of different ways of training GNNs using SSL. Specifically, we categorize SSL methods into contrastive and predictive models. In either category, we provide a unified framework for methods as well as how these methods differ in each component under the framework. Our unified treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms. We also summarize different SSL settings and the corresponding datasets used in each setting. To facilitate methodological development and empirical comparison, we develop a standardized testbed for SSL in GNNs, including implementations of common baseline methods, datasets, and evaluation metrics.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2022.3170559

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 38

  1. Yang, Rethinking the value of labels for improving class-imbalanced learning, 19290
  2. Xie, 2020, Noise2Same: Optimizing a self-supervised bound for image denoising, 20320
  3. Laine, 2019, High-quality self-supervised deep image denoising, Adv. Neural Inf. Process. Syst., 32, 6970
  4. Batson, Noise2Self: Blind denoising by self-supervision, 524
  5. Hassani, Contrastive multi-view representation learning on graphs, 4116
  6. Chen, A simple framework for contrastive learning of visual representations, 1597
  7. Kipf, Semi-supervised classification with graph convolutional networks
  8. Xu, 2019, How powerful are graph neural networks?, Int. Conf. Learn. Representations
  9. Yuan, StructPool: Structured graph pooling via conditional random fields
  10. Veličković, 2019, Deep graph infomax, Int. Conf. Learn. Representations
  11. Sun, Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization
  12. Hu, Strategies for pre-training graph neural networks
  13. You, Graph contrastive learning with augmentations, 5812
  14. 10.1109/ICCV.2019.00662 · DOI
  15. Hasanzadeh, Semi-implicit graph variational auto-encoders, 10 712
  16. Rong, Self-supervised graph transformer on large-scale molecular data, 12559
  17. Zhang, From canonical correlation analysis to self-supervised graph neural networks, 76
  18. Ganin, Unsupervised domain adaptation by backpropagation, 1180
  19. Hjelm, Learning deep representations by mutual information estimation and maximization
  20. Belghazi, Mutual information neural estimation, 531
  21. Gutmann, Noise-contrastive estimation: A new estimation principle for unnormalized statistical models, 297
  22. Sohn, 2016, Improved deep metric learning with multi-class n-pair loss objective, 1857
  23. Suresh, Adversarial graph augmentation to improve graph contrastive learning, 15920
  24. Xu, InfoGCL: Information-aware graph contrastive learning, 30414
  25. Jin, Junction tree variational autoencoder for molecular graph generation, 2323
  26. Li, Dirichlet graph variational autoencoder, 5274
  27. Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1
  28. Zbontar, Barlow twins: Self-supervised learning via redundancy reduction, 12310
  29. Tishby, The information bottleneck method, 368
  30. Morris, Tudataset: A collection of benchmark datasets for learning with graphs
  31. Yang, Revisiting semi-supervised learning with graph embeddings, 40
  32. Zeng, GraphSAINT: Graph sampling based inductive learning method
  33. 10.5555/3454287.3455008 · doi:10.5555/3454287.3455008
  34. Fey, Fast graph representation learning with PyTorch geometric
  35. You, GraphRNN: Generating realistic graphs with deep auto-regressive models, 5708
  36. Shi, Graphaf: A flow-based autoregressive model for molecular graph generation
  37. Luo, Graphdf: A discrete flow model for molecular graph generation, 7192
  38. 10.24963/ijcai.2021/371 · doi:10.24963/ijcai.2021/371

Cited by

See all 389

Metadata

Authors
Xie, Y., Xu, Z., Zhang, J., Wang, Z., & Ji, S.
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2023-02-01
Volume
45
Issue
2
Pages
2412-2429
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W3129850062
Cited by
389
References
94
FWCI
26.33
Type
journal article
Language
en
Open access
green
Funders
  • National Science Foundation (IIS-2006861, DBI-2028361)
  • National Institutes of Health (1R21NS102828)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.