The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Self-Supervised Learning of Graph Neural Networks: A Unified Review
Open accessAbstract
Deep models trained in supervised mode have achieved remarkable success on a variety of tasks. When labeled samples are limited, self-supervised learning (SSL) is emerging as a new paradigm for making use of large amounts of unlabeled samples. SSL has achieved promising performance on natural language and image learning tasks. Recently, there is a trend to extend such success to graph data using graph neural networks (GNNs). In this survey, we provide a unified review of different ways of training GNNs using SSL. Specifically, we categorize SSL methods into contrastive and predictive models. In either category, we provide a unified framework for methods as well as how these methods differ in each component under the framework. Our unified treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms. We also summarize different SSL settings and the corresponding datasets used in each setting. To facilitate methodological development and empirical comparison, we develop a standardized testbed for SSL in GNNs, including implementations of common baseline methods, datasets, and evaluation metrics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
GAN（Generative Adversarial Nets）
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
BPR: Bayesian Personalized Ranking from Implicit Feedback
Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2618 · Cited by 4,242
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846
Research works printed but not linked: 39
- Yang, Rethinking the value of labels for improving class-imbalanced learning, 19290
- Xie, 2020, Noise2Same: Optimizing a self-supervised bound for image denoising, 20320
- Laine, 2019, High-quality self-supervised deep image denoising, Adv. Neural Inf. Process. Syst., 32, 6970
- Batson, Noise2Self: Blind denoising by self-supervision, 524
- Hassani, Contrastive multi-view representation learning on graphs, 4116
- Chen, A simple framework for contrastive learning of visual representations, 1597
- Kipf, Semi-supervised classification with graph convolutional networks
- Xu, 2019, How powerful are graph neural networks?, Int. Conf. Learn. Representations
- Yuan, StructPool: Structured graph pooling via conditional random fields
- Veličković, 2019, Deep graph infomax, Int. Conf. Learn. Representations
- Sun, Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization
- Hu, Strategies for pre-training graph neural networks
- You, Graph contrastive learning with augmentations, 5812
- 10.1109/ICCV.2019.00662 · DOI
- Hasanzadeh, Semi-implicit graph variational auto-encoders, 10 712
- Rong, Self-supervised graph transformer on large-scale molecular data, 12559
- Zhang, From canonical correlation analysis to self-supervised graph neural networks, 76
- Ganin, Unsupervised domain adaptation by backpropagation, 1180
- Hjelm, Learning deep representations by mutual information estimation and maximization
- Belghazi, Mutual information neural estimation, 531
- Gutmann, Noise-contrastive estimation: A new estimation principle for unnormalized statistical models, 297
- Sohn, 2016, Improved deep metric learning with multi-class n-pair loss objective, 1857
- Khosla, Supervised contrastive learning, 18661
- Suresh, Adversarial graph augmentation to improve graph contrastive learning, 15920
- Xu, InfoGCL: Information-aware graph contrastive learning, 30414
- Jin, Junction tree variational autoencoder for molecular graph generation, 2323
- Li, Dirichlet graph variational autoencoder, 5274
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1
- Zbontar, Barlow twins: Self-supervised learning via redundancy reduction, 12310
- Tishby, The information bottleneck method, 368
- Morris, Tudataset: A collection of benchmark datasets for learning with graphs
- Yang, Revisiting semi-supervised learning with graph embeddings, 40
- Zeng, GraphSAINT: Graph sampling based inductive learning method
- 10.5555/3454287.3455008 · doi:10.5555/3454287.3455008
- Fey, Fast graph representation learning with PyTorch geometric
- You, GraphRNN: Generating realistic graphs with deep auto-regressive models, 5708
- Shi, Graphaf: A flow-based autoregressive model for molecular graph generation
- Luo, Graphdf: A discrete flow model for molecular graph generation, 7192
- 10.24963/ijcai.2021/371 · doi:10.24963/ijcai.2021/371
Cited by
Self-supervised learning in medicine and healthcare
Krishnan, R., Rajpurkar, P., & Topol, E. J. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00914-1 · Cited by 605
Machine Learning Methods for Small Data Challenges in Molecular Science
Dou, B., Zhu, Z., Merkurjev, E., Ke, L., Chen, L., Jiang, J., Zhu, Y., Liu, J., Zhang, B., & Wei, G.-W. (2023) · Chemical Reviews · doi:10.1021/acs.chemrev.3c00189 · Cited by 499
Self-supervised learning for recommender systems: A survey
Yu, J., Yin, H., Xia, X., Chen, T., Li, J., & Huang, Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3282907 · Cited by 413
A Comprehensive Survey on Deep Graph Representation Learning
Ju, W., Fang, Z., Gu, Y., Liu, Z., Long, Q., Qiao, Z., Qin, Y., Shen, J., Sun, F., Xiao, Z., et al. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106207 · Cited by 341
Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387317 · Cited by 295
“The SSL methods can be roughly divided into the following categories: generative-based, contrastive-based, and adversarial-based methods.”
BackgroundSelf-citation
Metadata
- Authors
- Xie, Y., Xu, Z., Zhang, J., Wang, Z., & Ji, S.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2023-02-01
- Volume
- 45
- Issue
- 2
- Pages
- 2412-2429
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3129850062
- Cited by
- 389
- References
- 93
- FWCI
- 26.33
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- National Science Foundation (IIS-2006861, DBI-2028361)
- National Institutes of Health (1R21NS102828)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Self-Supervised Learning of Graph Neural Networks: A Unified Review. https://app.cochat.ai/papers/10.1109/tpami.2022.3170559
- The paper
- Xie, Y., Xu, Z., Zhang, J., Wang, Z., & Ji, S. (2023). Self-Supervised Learning of Graph Neural Networks: A Unified Review. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2022.3170559
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
