The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Deep Neural Networks for Learning Graph Representations
Open accessAbstract
In this paper, we propose a novel model for learning graph representations, which generates a low-dimensional vector representation for each vertex by capturing the graph structural information. Different from other previous research efforts, we adopt a random surfing model to capture graph structural information directly, instead of using the sampling-based method for generating linear sequences proposed by Perozzi et al. (2014). The advantages of our approach will be illustrated from both theorical and empirical perspectives. We also give a new perspective for the matrix factorization method proposed by Levy and Goldberg (2014), in which the pointwise mutual information (PMI) matrix is considered as an analytical solution to the objective function of the skip-gram model with negative sampling proposed by Mikolov et al. (2013). Unlike their approach which involves the use of the SVD for finding the low-dimensitonal projections from the PMI matrix, however, the stacked denoising autoencoder is introduced in our model to extract complex features and model non-linearities. To demonstrate the effectiveness of our model, we conduct experiments on clustering and visualization tasks, employing the learned vertex representations as features. Empirical results on datasets of varying sizes show that our model outperforms other stat-of-the-art models in such tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2008) · Journal of Machine Learning Research · Cited by 33,051
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
(2007) · doi:10.5555/1283383.1283494 · Cited by 7,364
LINE: Large-scale Information Network Embedding
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., & Mei, Q. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741093 · Cited by 5,416 · cited 2×
“We conducted visualization experiments by following (Tang et al. 2015 ) to compare the performance of DNGR with SVD, deepwalk and SGNS on Wine.”
MethodGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 1×
“One effective solution, proposed in (Hinton and Salakhutdinov 2006; Bengio et al. 2007), is to use the greedy layer-wise unsupervised pre-training.”
BackgroundThe Approximation of One Matrix by Another of Lower Rank
Eckart, C., & Young, G. (1936) · Psychometrika · doi:10.1007/bf02288367 · Cited by 3,800
Word association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 2×
“Deep neural networks, which can be used to learn multiple levels of feature representations, has achieved successful results in different fields (Dahl et al. 2012; Krizhevsky, Sutskever, and Hinton 2012).”
BackgroundProducing high-dimensional semantic spaces from lexical co-occurrence
Lund, K., & Burgess, C. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204766 · Cited by 1,811
Contextual correlates of semantic similarity
Miller, G. A., & Charles, W. G. (1991) · Language and Cognitive Processes · doi:10.1080/01690969108406936 · Cited by 1,594
Neural Word Embedding as Implicit Matrix Factorization
Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,582
GraRep: Learning Graph Representations with Global Structural Information
Cao, S., Lu, W., & Xu, Q. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806512 · Cited by 1,571
Self-citationAuto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,345
Improving Distributional Similarity with Lessons Learned from Word Embeddings
Levy, O., Goldberg, Y., & Dagan, I. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00134 · Cited by 1,308
(1993) · Cited by 1,173
DeepWalk: Online Learning of Social Representations
Perozzi, B., Al‐Rfou, R., & Skiena, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.6652 · Cited by 1,107
Word association norms, mutual information, and lexicography
Church, K. W., & Hanks, P. (1989) · Proceedings of the 27th annual meeting on Association for Computational Linguistics - · doi:10.3115/981623.981633 · Cited by 1,094
Improving Word Representations via Global Context and Multiple Word Prototypes
Huang, E., Socher, R., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,073
Placing search in context: the concept revisited.
Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., & Ruppin, E. (2002) · Cited by 961
A study on similarity and relatedness using distributional and WordNet-based approaches
Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., Paşca, M., & Soroa, A. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics on - NAACL '09 · doi:10.3115/1620754.1620758 · Cited by 860
Placing search in context: the concept revisited
Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., & Ruppin, E. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372094 · Cited by 839
Extracting semantic representations from word co-occurrence statistics: A computational study
Bullinaria, J. A., & Levy, J. P. (2007) · Behavior Research Methods · doi:10.3758/bf03193020 · Cited by 805
Relational learning via latent social dimensions
Tang, L., & Liu, H. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557109 · Cited by 799 · cited 1×
“The learned vertex representations were shown to be effective across a few tasks, outperforming several previous approaches such as spectral clustering and modularity method (Tang and Liu 2009a; 2009b; 2011; Macskassy and Provost 2003).”
MethodSignificance Testing of the Spearman Rank Correlation Coefficient
Zar, J. H. (1972) · Journal of the American Statistical Association · doi:10.1080/01621459.1972.10481251 · Cited by 689
Impact of Similarity Measures on Web-page Clustering
Strehl, A. L., Ghosh, J., & Mooney, R. J. (2000) · Cited by 658
Learning Deep Representations for Graph Clustering
Tian, F., Gao, B., Cui, Q., Chen, E., & Liu, T.-Y. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8916 · Cited by 657
LINE: Large-Scale Information Network Embedding
Tang, J. G. (2024) · doi:10.57702/6e6l3h4g · Cited by 627
Gutmann, M. U., & Hyvärinen, A. (2012) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5555/2503308.2188396 · Cited by 581
Significance Testing of the Spearman Rank Correlation Coefficient
Zar, J. H. (1972) · Journal of the American Statistical Association · doi:10.2307/2284441 · Cited by 396
Leveraging social media networks for classification
Tang, L., & Liu, H. (2011) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0210-x · Cited by 360
A Fast and Simple Algorithm for Training Neural Probabilistic Language Models
Mnih, A., & Teh, Y. W. (2012) · arXiv · doi:10.48550/arxiv.1206.6426 · Cited by 315
Bullinaria, J. A., & Levy, J. P. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0183-8 · Cited by 303
A Simple Relational Classifier
Macskassy, S. A., & Provost, F. (2003) · doi:10.21236/ada452802 · Cited by 255
Scalable learning of collective behavior based on sparse social dimensions
Tang, L., & Liu, H. (2009) · Proceedings of the 18th ACM conference on Information and knowledge management · doi:10.1145/1645953.1646094 · Cited by 233 · cited 1×
“The learned vertex representations were shown to be effective across a few tasks, outperforming several previous approaches such as spectral clustering and modularity method (Tang and Liu 2009a; 2009b; 2011; Macskassy and Provost 2003).”
MethodUsing wiktionary for computing semantic relatedness
Zesch, T., Müller, C., & Gurevych, I. (2008) · TUbilio (Technical University of Darmstadt) · Cited by 150
Cited by
A Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications
Cai, H., Zheng, V. W., & Chang, K. C.-C. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2807452 · Cited by 2,034
Graph embedding techniques, applications, and performance: A survey
Goyal, P., & Ferrara, E. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.03.022 · Cited by 1,801
“The growing research on deep learning has led to a deluge of deep neural networks based methods applied to graphs [23], [33], [34].”
BackgroundDeep Learning on Graphs: A Survey
Zhang, Z., Cui, P., & Zhu, W. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981333 · Cited by 1,633
Cui, P., Wang, X., Pei, J., & Zhu, W. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2849727 · Cited by 1,274
Metadata
- Authors
- Cao, S., Lu, W., & Xu, Q.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2016-02-21
- Volume
- 30
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2415243320
- Cited by
- 1,014
- References
- 41
- FWCI
- 43.93
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.