The DOI matches its Crossref record; 142 of the 142 references it cites were found in a registry or an index.
Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive
Open accessAbstract
Deep learning on graphs has recently achieved remarkable success on a variety of tasks, while such success relies heavily on the massive and carefully labeled data. However, precise annotations are generally very expensive and time-consuming. To address this problem, self-supervised learning (SSL) is emerging as a new paradigm for extracting informative knowledge through well-designed pretext tasks without relying on manual labels. In this survey, we extend the concept of SSL, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of existing SSL techniques for graph data. Specifically, we divide existing graph SSL methods into three categories: contrastive, generative, and predictive. More importantly, unlike other surveys that only provide a high-level description of published research, we present an additional mathematical summary of existing works in a unified framework. Furthermore, to facilitate methodological development and empirical comparisons, we also summarize the commonly used datasets, evaluation metrics, downstream tasks, open-source implementations, and experimental study of various algorithms. Finally, we discuss the technical challenges and potential future directions for improving graph self-supervised learning. Latest advances in graph SSL are summarized in a GitHub repositoryhttps://github.com/LirongWu/awesome-graph-self-supervised-learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
(1967) · Cited by 23,719
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Fast unfolding of communities in large networks
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2008/10/p10008 · Cited by 21,424
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
(1999) · Cited by 12,854
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
node2vec: Scalable Feature Learning for Networks
Grover, A., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939754 · Cited by 10,926
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 8,657
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678 · cited 1×
“Though numerous strategies have been proposed to address this problem in the CV and NLP domains [90], they are difficult to apply directly to graphs due to the inherent non-Euclidean structure of graphs.”
BackgroundWeiss, K., Khoshgoftaar, T. M., & Wang, D. (2016) · Journal of Big Data · doi:10.1186/s40537-016-0043-6 · Cited by 6,260
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635
Targeted Branching for the Maximum Independent Set Problem Using Graph Neural Networks
Silva, G., Rodrigues, M., Teixeira, A., & Amorim, M. (2024) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.sea.2024.20 · Cited by 5,183
Power and Centrality: A Family of Measures
Bonacich, P. (1987) · American Journal of Sociology · doi:10.1086/228631 · Cited by 5,165
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
The MovieLens Datasets: History and Context
Harper, F. M., & Konstan, J. A. (2016) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2827872 · Cited by 3,907
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
Collective Classification in Network Data
Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., & Eliassi‐Rad, T. (2008) · AI Magazine · doi:10.1609/aimag.v29i3.2157 · Cited by 3,293
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. (2020) · UTS ePRESS (University of Technology Sydney) · Cited by 3,174
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. U., & Hyvärinen, A. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 2,989
Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning
Li, Q., Han, Z., & Wu, X.-M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11604 · Cited by 2,872
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09882 · Cited by 2,499
Quantum chemistry structures and properties of 134 kilo molecules
Ramakrishnan, R., Dral, P. O., Rupp, M., & von Lilienfeld, O. A. (2014) · Scientific Data · doi:10.1038/sdata.2014.22 · Cited by 1,966
Identifying Resilient Communities in Road Networks: A Path-Based Embedding Approach
Wagner, C., Dodge, S., & Alizadeh, D. (2025) · Leibniz international proceedings in informatics · doi:10.4230/lipics.giscience.2025.9 · Cited by 1,543
The SIDER database of drugs and side effects
Kuhn, M., Letunic, I., Jensen, L. J., & Bork, P. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1075 · Cited by 1,451
Automating the Construction of Internet Portals with Machine Learning
McCallum, A. K., Nigam, K., Rennie, J., & Seymore, K. (2000) · Information Retrieval · doi:10.1023/a:1009953814988 · Cited by 1,280
Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification
Chen, W., Chen, X., Zhang, J., & Huang, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.145 · Cited by 1,224
Contrastive Learning with Hard Negative SamplesPreprint
Robinson, J., Chuang, C.-Y., Sra, S., & Jegelka, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04592 · Cited by 1,217
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Nowozin, S., Cseke, B., & Tomioka, R. (2016) · arXiv · doi:10.48550/arxiv.1606.00709 · Cited by 1,183
Protein function prediction via graph kernels
Borgwardt, K. M., Ong, C. S., Schonauer, S., Vishwanathan, S. V. N., Smola, A. J., & Kriegel, H.-P. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti1007 · Cited by 1,105
Self-supervised Learning: Generative or Contrastive
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3090866 · Cited by 1,087
DeepTox: toxicity prediction using deep learning
Mayr, A., Klambauer, G., Unterthiner, T., & Hochreiter, S. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00080 · Cited by 1,081
Graph Contrastive Learning with Adaptive Augmentation
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., & Wang, L. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449802 · Cited by 1,066
Big data and its technical challenges.
Jagadish, H. V., Gehrke, J., Labrinidis, A., Papakonstantinou, Y., Patel, J. M., Ramakrishnan, R., & Shahabi, C. (2014) · Communications of the ACM · doi:10.1145/2611567 · Cited by 1,048
CiteSeer: an automatic citation indexing system
Giles, C. L., Bollacker, K. D., & Lawrence, S. (1998) · Proceedings of the third ACM conference on Digital libraries - DL '98 · doi:10.1145/276675.276685 · Cited by 1,040
Asymptotic evaluation of certain markov process expectations for large time, II
Donsker, M. D., & Varadhan, S. R. S. (1975) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.3160280206 · Cited by 1,025
Sun, F.-Y., Hoffmann, J., Verma, V., & Tang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01000 · Cited by 1,025
Yanardag, P., & Vishwanathan, S. V. N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783417 · Cited by 1,021
A simple framework for contrastive learning of visual representations
Chen, T. (2024) · TIB Data Manager · doi:10.57702/iuuvgtaz · Cited by 973
Mutual Information Neural Estimation.
Belghazi, M. I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A. C., & Hjelm, D. (2018) · International Conference on Machine Learning · Cited by 920
Joint Unsupervised Learning of Deep Representations and Image Clusters
Yang, J., Parikh, D., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.556 · Cited by 879
Debnath, A. K., Lopez de Compadre, R. L., Debnath, G., Shusterman, A. J., & Hansch, C. (1991) · Journal of Medicinal Chemistry · doi:10.1021/jm00106a046 · Cited by 868
Graph Contrastive Learning with Augmentations
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., & Shen, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13902 · Cited by 824
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
Qiu, J., Chen, Q., Dong, Y., Zhang, J., Yang, H., Ding, M., Wang, K., & Tang, J. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403168 · Cited by 777
Variational Graph Auto-Encoders
Kipf, T., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07308 · Cited by 772
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
Žbontar, J., Jing, L., Misra, I., LeCun, Y., & Deny, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03230 · Cited by 751
ToxCast Chemical Landscape: Paving the Road to 21st Century Toxicology
Richard, A. M., Judson, R. S., Houck, K. A., Grulke, C. M., Volarath, P., Thillainadarajah, I., Yang, C., Rathman, J., Martin, M. T., Wambaugh, J. F., et al. (2016) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.6b00135 · Cited by 709
Derivation and Validation of Toxicophores for Mutagenicity Prediction
Kazius, J., McGuire, R., & Bursi, R. (2005) · Journal of Medicinal Chemistry · doi:10.1021/jm040835a · Cited by 666
Rival penalized competitive learning for clustering analysis, RBF net, and curve detection
Xu, L., Krzyzak, A., & Oja, E. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.238318 · Cited by 613
Shchur, O., Mumme, M., Bojchevski, A., & Günnemann, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05868 · Cited by 588
Distinguishing Enzyme Structures from Non-enzymes Without Alignments
Dobson, P. D., & Doig, A. J. (2003) · Journal of Molecular Biology · doi:10.1016/s0022-2836(03)00628-4 · Cited by 579
Graph Representation Learning via Graphical Mutual Information Maximization
Peng, Z., Huang, W., Luo, M., Zheng, Q., Rong, Y., Xu, T., & Huang, J. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380112 · Cited by 536
Predicting multicellular function through multi-layer tissue networks
Zitnik, M., & Leskovec, J. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx252 · Cited by 519
Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation
Yu, J., Yin, H., Li, J., Wang, Q., Hung, N. Q. V., & Zhang, X. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449844 · Cited by 519
Graph Self-Supervised Learning: A Survey
Liu, Y., Jin, M., Pan, S., Zhou, C., Zheng, Y., Xia, F., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3172903 · Cited by 510
Towards Deeper Graph Neural Networks
Liu, M., Gao, H., & Ji, S. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403076 · Cited by 507
IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning
Riesen, K., & Bunke, H. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89689-0_33 · Cited by 498
Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning
Wang, X., Liu, N., Han, H., & Shi, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467415 · Cited by 467
GPT-GNN: Generative Pre-Training of Graph Neural Networks
Hu, Z., Dong, Y., Wang, K., Chang, K.-W., & Sun, Y. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403237 · Cited by 446
Graph-Bert: Only Attention is Needed for Learning Graph RepresentationsPreprint
Zhang, J., Zhang, H., Xia, C., & Li, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05140 · Cited by 437
Self-citationContrastive Multi-View Representation Learning on Graphs
Hassani, K., & Khasahmadi, A. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05582 · Cited by 423
MGAE: Marginalized Graph Autoencoder for Graph Clustering
Wang, C., Pan, S., Long, G., Zhu, X., & Jiang, J. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132967 · Cited by 420
Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural NetworksPreprint
Mernyei, P., & Cangea, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02901 · Cited by 411
Deep Graph Contrastive Representation Learning
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04131 · Cited by 389
Self-Supervised Learning of Graph Neural Networks: A Unified Review
Xie, Y., Xu, Z., Zhang, J., Wang, Z., & Ji, S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3170559 · Cited by 389
Self-Supervised Graph Transformer on Large-Scale Molecular Data
Rong, Y., Bian, Y., Xu, T., Xie, W., Wei, Y., Huang, W., & Huang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02835 · Cited by 388
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2020) · arXiv · doi:10.48550/arxiv.2011.10566 · Cited by 381
How Powerful are Graph Neural Networks?
Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00826 · Cited by 332
A Bayesian Approach to in Silico Blood-Brain Barrier Penetration Modeling
Martins, I. F., Teixeira, A. L., Pinheiro, L., & Falcao, A. O. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci300124c · Cited by 331
Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning
Park, J., Lee, M., Chang, H. J., Lee, K., & Choi, J. Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00662 · Cited by 331
Deep Spectral Clustering Using Dual Autoencoder Network
Yang, X., Deng, C., Zheng, F., Yan, J., & Liu, W. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00419 · Cited by 313
Pre-training of Graph Augmented Transformers for Medication Recommendation
Shang, J., Ma, T., Xiao, C., & Sun, J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/825 · Cited by 311
Unsupervised Attributed Multiplex Network Embedding
Park, C., Kim, D., Han, J., & Yu, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5985 · Cited by 286
Computational Modeling of β-Secretase 1 (BACE-1) Inhibitors Using Ligand Based Approaches
Subramanian, G., Ramsundar, B., Pande, V., & Denny, R. A. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00290 · Cited by 285
Veličković, P., Fedus, W., Hamilton, W. L., Lió, P., Bengio, Y., & Hjelm, R. D. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.40744 · Cited by 279
Self-supervised Learning on Graphs: Deep Insights and New DirectionPreprint
Jin, W., Derr, T., Liu, H., Wang, Y., Wang, S., Liu, Z., & Tang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10141 · Cited by 261
Hard Negative Mixing for Contrastive Learning
Kalantidis, Y., Sarıyıldız, M. B., Pion, N., Weinzaepfel, P., & Larlus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01028 · Cited by 257
On Mutual Information Maximization for Representation Learning
Tschannen, M., Djolonga, J., Rubenstein, P. K., Gelly, S., & Lučić, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13625 · Cited by 253
How to Find Your Friendly Neighborhood: Graph Attention Design with Self-SupervisionPreprint
Kim, D., & Oh, A. H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.04879 · Cited by 247
Sun, K., Lin, Z., & Zhu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6048 · Cited by 237
Socially-Aware Self-Supervised Tri-Training for Recommendation
Yu, J., Yin, H., Gao, M., Xia, X., Zhang, X., & Viet Hung, N. Q. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467340 · Cited by 228
Huang, R., Xia, M., Nguyen, D.-T., Zhao, T., Sakamuru, S., Zhao, J., Shahane, S. A., Rossoshek, A., & Simeonov, A. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00085 · Cited by 227
Comparison of Descriptor Spaces for Chemical Compound Retrieval and Classification
Wale, N., & Karypis, G. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.39 · Cited by 226
SLAPS: Self-Supervision Improves Structure Learning for Graph Neural NetworksPreprint
Fatemi, B., Asri, L. E., & Kazemi, S. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05034 · Cited by 216
Contrastive Learning of Structured World ModelsPreprint
Kipf, T., van der Pol, E., & Welling, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.12247 · Cited by 200
Non-Local Graph Neural Networks
Liu, M., Wang, Z., & Ji, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3134200 · Cited by 196
Contrastive Self-supervised Learning for Graph Classification
Zeng, J., & Xie, P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17293 · Cited by 194
Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning
Wan, S., Pan, S., Yang, J. J., & Gong, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17206 · Cited by 190
HDMI: High-order Deep Multiplex Infomax
Jing, B., Park, C., & Tong, H. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449971 · Cited by 180
Sub-Graph Contrast for Scalable Self-Supervised Graph Representation Learning
Jiao, Y., Xiong, Y., Zhang, J., Zhang, Y., Zhang, T., & Zhu, Y. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00031 · Cited by 176
Sun, Q., Li, J., Peng, H., Wu, J., Ning, Y., Yu, P. S., & He, L. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449822 · Cited by 162
When Does Self-Supervision Help Graph Convolutional Networks?
You, Y., Chen, T., Wang, Z., & Shen, Y. (2020) · PubMed · doi:10.48550/arxiv.2006.09136 · Cited by 155
N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embedding
McConville, R., Santos‐Rodríguez, R., Piechocki, R. J., & Craddock, I. J. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413131 · Cited by 148
Heterogeneous Deep Graph InfomaxPreprint
Ren, Y., Лю, Б., Huang, C., Dai, P., Bo, L., & Zhang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08538 · Cited by 136
Bipartite Graph Embedding via Mutual Information Maximization
Cao, J., Lin, X., Guo, S., Liu, L., Liu, T., & Wang, B. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441783 · Cited by 133
Novick, P. A., Ortiz, O. F., Poelman, J., Abdulhay, A., & Pande, V. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0079568 · Cited by 130
Unsupervised Streaming Feature Selection in Social Media
Li, J., Hu, X., Tang, J., & Liu, H. (2015) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2806416.2806501 · Cited by 127
Noise2Same: Optimizing A Self-Supervised Bound for Image DenoisingPreprint
Xie, Y., Wang, Z., & Ji, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11971 · Cited by 119
Bootstrapped Representation Learning on GraphsPreprint
Thakoor, S., Tallec, C., Azar, M. G., Munos, R., Veličković, P., & Valko, M. (2021) · arXiv (Cornell University) · Cited by 118
Motif-Driven Contrastive Learning of Graph Representations
Zhang, S., Hu, Z., Subramonian, A., & Sun, Y. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3364059 · Cited by 108
Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation
Hao, B., Zhang, J., Yin, H., Li, C., & Chen, H. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441738 · Cited by 101
Self-Supervised Graph Representation Learning via Global Context PredictionPreprint
Peng, Z., Dong, Y., Luo, M., Wu, X.-M., & Zheng, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01604 · Cited by 94
Mavromatis, C., & Karypis, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06946 · Cited by 87
Towards Domain-Agnostic Contrastive LearningPreprint
Verma, V., Luong, M.-T., Kawaguchi, K., Pham, H., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04419 · Cited by 76
Hop-Count Based Self-supervised Anomaly Detection on Attributed Networks
Huang, T., Pei, Y., Menkovski, V., & Pechenizkiy, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26387-3_14 · Cited by 73
Effectiveness of 2D Fingerprints for Scaffold Hopping
Gardiner, E. J., Holliday, J. D., O’Dowd, C., & Willett, P. (2011) · Future Medicinal Chemistry · doi:10.4155/fmc.11.4 · Cited by 66
Transfer Learning of Graph Neural Networks with Ego-graph Information MaximizationPreprint
Zhu, Q., Yang, C., Xu, Y., Wang, H., Zhang, C., & Han, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05204 · Cited by 66
Self-Supervised Learning of Contextual Embeddings for Link Prediction in Heterogeneous Networks
Wang, P., Agarwal, K., Ham, C., Choudhury, S., & Reddy, C. K. A. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450060 · Cited by 61
Self-supervised learning: Generative or contrastivePreprint
Xiao, X. (2024) · arXiv (Cornell University) · doi:10.57702/ht8rsma4 · Cited by 55
Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous GraphsPreprint
Hwang, D., Park, J., Kwon, S., Kim, K.-M., Ha, J.-W., & Kim, H. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08294 · Cited by 53
Towards Robust Graph Contrastive LearningPreprint
Jovanović, N., Zhao, M., Faber, L., & Wattenhofer, R. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.13085 · Cited by 47
Iterative Graph Self-Distillation
Zhang, H., Lin, S., Liu, W., Zhou, P., Tang, J., Liang, X., & Xing, E. P. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3303885 · Cited by 47
RankMI: A Mutual Information Maximizing Ranking Loss
Kemertas, M., Pishdad, L., Derpanis, K. G., & Fazly, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01437 · Cited by 45
Self-supervised Training of Graph Convolutional NetworksPreprint
Zhu, Q., Du, B., & Yan, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02380 · Cited by 44
Pre-Training Graph Neural Networks for Generic Structural Feature ExtractionPreprint
Hu, Z., Fan, C., Chen, T., Chang, K.-W., & Sun, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13728 · Cited by 43
Spatio-Temporal Deep Graph InfomaxPreprint
Opolka, F. L., Solomon, A., Cangea, C., Veličković, P., Liò, P., & Hjelm, R. D. (2019) · arXiv (Cornell University) · doi:10.17863/cam.40745 · Cited by 39
Self-supervised Graph Neural Networks without explicit negative samplingPreprint
Kefato, Z. T., & Girdzijauskas, Š. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14958 · Cited by 37
Graph-based neural network models with multiple self-supervised auxiliary tasks
Manessi, F., & Rozza, A. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.04.021 · Cited by 34
CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation LearningPreprint
Zhu, Y., Xu, Y., Yu, F., Wu, S., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01674 · Cited by 29
Label Contrastive Coding Based Graph Neural Network for Graph Classification
Ren, Y., Bai, J., & Zhang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-73194-6_10 · Cited by 29
Pre-training on dynamic graph neural networks
Chen, K., Zhang, J., Jiang, L., Wang, Y., & Dai, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.070 · Cited by 24
Learning graph representation by aggregating subgraphs via mutual information maximization
Liu, Z., Wang, C., Han, C., & Guo, T. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126392 · Cited by 22
Calibrating and Improving Graph Contrastive LearningPreprint
Ma, K., Yang, H., Yang, H. Q., Chen, Y., & Cheng, J. S.-C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.11525 · Cited by 14
GraphMS: Drug Target Prediction Using Graph Representation Learning with Substructures
Cheng, S., Zhang, L., Jin, B., Zhang, Q., Lu, X., You, M., & Tian, X. (2021) · Applied Sciences · doi:10.3390/app11073239 · Cited by 14
Consistent Representation Learning for High Dimensional Data AnalysisPreprint
Li, S. Z., Wu, L., & Zang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00481 · Cited by 10
Self-citationSelf-supervised edge features for improved Graph Neural Network trainingPreprint
Sehanobish, A., Ravindra, N. G., & van Dijk, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04777 · Cited by 7
COAD: Contrastive Pre-training with Adversarial Fine-tuning for Zero-shot Expert Linking.Preprint
Chen, B., Zhang, J., Zhang, X., Tang, X., Cai, L., Hong, C., Li, C., Zhang, P., & Tang, J. (2020) · arXiv (Cornell University) · Cited by 4
Deep Clustering and Representation Learning that Preserves Geometric StructuresPreprint
Wu, L., Liu, Z., Zang, Z., Xia, J., Li, S., & Li, S. Z. (2021) · arXiv (Cornell University) · Cited by 3
Self-citationChen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Research works printed but not linked: 4
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1
- Sohn, Improved deep metric learning with multi-class N-pair loss objective, 1857
- Xie, Unsupervised deep embedding for clustering analysis, 478
- Chuang, Debiased contrastive learning
Cited by
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P.-A., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 538
Self-citationSelf-supervised learning for recommender systems: A survey
Yu, J., Yin, H., Xia, X., Chen, T., Li, J., & Huang, Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3282907 · Cited by 413
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122807 · Cited by 410
Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387317 · Cited by 295
“The SSL methods can be roughly divided into the following categories: generative-based, contrastive-based, and adversarial-based methods.”
BackgroundDeep learning methods for molecular representation and property prediction
Li, Z., Jiang, M., Wang, S., & Zhang, S. (2022) · Drug Discovery Today · doi:10.1016/j.drudis.2022.103373 · Cited by 279
Metadata
- Authors
- Wu, L., Lin, H., Tan, C., Gao, Z., & Li, S. Z.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2023-04-01
- Volume
- 35
- Issue
- 4
- Pages
- 4216-4235
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W3204453541
- Cited by
- 290
- References
- 142
- FWCI
- 14.11
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.