The DOI matches its DataCite record; 18 of the 19 references it cites were found in a registry or an index.
Distributed Representations of Words and Phrases and their Compositionality
Open accessAbstract
The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the training speed. By subsampling of the frequent words we obtain significant speedup and also learn more regular word representations. We also describe a simple alternative to the hierarchical softmax called negative sampling. An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of "Canada" and "Air" cannot be easily combined to obtain "Air Canada". Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Earlier word embedding models, e.g., word2vec [60] and GloVe [61], predominantly relied on syntactic contexts for training.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [60]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Contrary to string similarity-based approaches, continuous word representations capture not only lexical but also semantic properties of tokens.”
From A Method for the Calculation of Parallel Passages for Buddhist Chinese Sources Based on Million-scale Nearest Neighbor Search (2020) · cites this paper as Mikolov et al. (2013)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Mikolov et al. (2013) demonstrated that unweighted averaging of word embeddings performs well for the representation of short phrases.”
From A Method for the Calculation of Parallel Passages for Buddhist Chinese Sources Based on Million-scale Nearest Neighbor Search (2020) · cites this paper as Mikolov et al. (2013)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even during this stage, there was some use of pre-trained LMs exemplified by word2vec[85,86] and GloVe[95], but they were used for only a limited portion of the final model parameters.”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [86]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“These models were trained on vast amounts of unlabeled data, aiming to capture intricate grammatical and semantic rules.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [60]
The cited source says what the claim attributes to it.
In the citing paper
“In the next steps we filter out stopwords (e. g. 'a' , 'and' , 'the') and build a Word2Vec model (Mikolov et al., 2013) from these queries to get a model of the occurrence and co-occurrence of terms in these queries.”
From AI art in architecture (2023) · cites this paper as Mikolov et al., 2013
The cited source says what the claim attributes to it.
In the citing paper
“Negative sampling has been proved quite effective for both learning knowledge graph embedding (Trouillon et al., 2016) and word embedding (Mikolov et al., 2013).”
From RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space (2019) · cites this paper as Mikolov et al., 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Self-citationA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,949
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913
Self-citation(2010) · Cited by 1,906
Extensions of recurrent neural network language model
Mikolov, T., Kombrink, S., Burget, L., Černocký, J., & Khudanpur, S. P. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947611 · Cited by 1,622
Self-citationDomain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Glorot, X., Bordes, A., & Bengio, Y. (2011) · Cited by 1,569
Semantic Compositionality through Recursive Matrix-Vector Spaces
Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,262
(2011) · Cited by 1,192
Research works printed but not linked: 3
- [1][1] Yoshua Bengio, R´ ejean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. The Journal of Machine Learning Research , 3:1137-1155, 2003. -
- [7][7] Tomas Mikolov. Statistical Language Models Based on Neural Networks. PhD thesis, PhD Thesis, Brno University of Technology , 2012. -
- [13][13] David E Rumelhart, Geoffrey E Hintont, and Ronald J Williams. Learning representations by backpropagating errors. Nature , 323(6088):533-536, 1986. -
Cited by
Sorted by citations
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Self-citationConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Self-citationDistributed Representations of Sentences and DocumentsPreprint
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 7,905
Self-citationBioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
“Though it could be significantly complex and costly, most recent works done in XAI field belong to post-hoc class, it includes natural language explanations [71], visualizations of learned models [72], and explanations by example [73].”
Background(2013) · Cited by 5,726
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
Modeling Relational Data with Graph Convolutional Networks
Schlichtkrull, M., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_38 · Cited by 5,570
LINE: Large-scale Information Network Embedding
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., & Mei, Q. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741093 · Cited by 5,416
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.01626 · Cited by 4,481
Neural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
Knowledge Graph Embedding by Translating on Hyperplanes
Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8870 · Cited by 3,915
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Learning Entity and Relation Embeddings for Knowledge Graph Completion
Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9491 · Cited by 3,749
Graph Convolutional Neural Networks for Web-Scale Recommender Systems
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219890 · Cited by 3,733
Self-citationUnsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664
Geometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
Supervised Contrastive Learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., & Krishnan, D. (2020) · arXiv · doi:10.48550/arxiv.2004.11362 · Cited by 3,569
Deep Neural Networks for YouTube Recommendations
Covington, P., Adams, J., & Sargin, E. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959190 · Cited by 3,539
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342
“…proposed several DNN architectures to solve natural language processing (NLP) tasks such as machine translation (Sutskever et al., 2014; Bahdanau et al., 2015), learning word embeddings (Mikolov et al., 2013; Mikolov et al., 2013) and document classification (Le and Mikolov, 2014; Goldberg, 2016).”
BackgroundLearning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
SimCSE: Simple Contrastive Learning of Sentence Embeddings
Gao, T., Yao, X., & Chen, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.552 · Cited by 2,851
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 2,683
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Knowledge Graph Embedding: A Survey of Approaches and Applications
Wang, Q., Mao, Z., Wang, B., & Guo, L. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2754499 · Cited by 2,654
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357895 · Cited by 2,558
Deep learning for visual understanding: A review
Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.09.116 · Cited by 2,504
Film: Visual reasoning with a general conditioning layer
Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11671 · Cited by 2,490
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456
Attention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450
Recurrent Convolutional Neural Networks for Text Classification
Lai, S., Xu, L., Liu, K., & Zhao, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9513 · Cited by 2,384
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329
The COVID-19 social media infodemic
Cinelli, M., Quattrociocchi, W., Galeazzi, A., Valensise, C. M., Brugnoli, E., Schmidt, A. L., Zola, P., Zollo, F., & Scala, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-73510-5 · Cited by 2,313
DeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,309
Self-citationDong, Y., Chawla, N. V., & Swami, A. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098036 · Cited by 2,295
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270
Embedding Entities and Relations for Learning and Inference in Knowledge Bases
Yang, B., Yih, W.-T., He, X., Gao, J., & Deng, L. (2014) · arXiv · doi:10.48550/arxiv.1412.6575 · Cited by 2,252
Graph Convolutional Networks for Text Classification
Yao, L., Mao, C., & Luo, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017370 · Cited by 2,224
Deep learning for sentiment analysis: A survey
Zhang, L., Wang, S., & Liu, B. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1253 · Cited by 2,182
Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification
Zhou, P., Shi, W. Y., Tian, J., Qi, Z., Li, B., Hao, H., & Xu, B. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-2034 · Cited by 2,147
Unsupervised Data Augmentation for Consistency TrainingPreprint
Xie, Q., Dai, Z., Hovy, E. H., Luong, T., & Le, Q. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12848 · Cited by 2,098
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications
Cai, H., Zheng, V. W., & Chang, K. C.-C. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2807452 · Cited by 2,034
Named Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J., & Nichols, E. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00104 · Cited by 2,007
Self-supervised Learning: Generative or Contrastive
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3090866 · Cited by 2,008
CatBoost for big data: an interdisciplinary review
Hancock, J., & Khoshgoftaar, T. M. (2020) · Journal Of Big Data · doi:10.1186/s40537-020-00369-8 · Cited by 2,023
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977
“Previously word2vec (Mikolov et al., 2013a;b) shows that linear relationships between word vectors emerge as a result of training them to predict adjacent words in sentences and paragraphs.”
BackgroundSession-Based Recommendation with Graph Neural Networks
Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., & Tan, T. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301346 · Cited by 1,958
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Pérez, M., Wang, G., Duan, L., & Kot, A. C. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,917
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,869
Text Classification Algorithms: A Survey
Kowsari, K., Meimandi, K. J., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. E. (2019) · Information · doi:10.3390/info10040150 · Cited by 1,866
A Fast and Accurate Dependency Parser using Neural Networks
Chen, D., & Manning, C. D. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1082 · Cited by 1,844
word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding methodPreprint
Goldberg, Y., & Levy, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.3722 · Cited by 1,837
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
Natural language processing: state of the art, current trends and challenges
Khurana, D., Koli, A., Khatter, K., & Singh, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13428-4 · Cited by 1,823
Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs
Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., & Bronstein, M. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.576 · Cited by 1,820
Open Graph Benchmark: Datasets for Machine Learning on GraphsPreprint
Hu, W., Fey, M., Žitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., & Leskovec, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00687 · Cited by 1,788
Pre-Training With Whole Word Masking for Chinese BERT
Cui, Y., Che, W., Liu, T., Qin, B., & Yang, Z. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3124365 · Cited by 1,783
Self-supervised Graph Learning for Recommendation
Wu, J., Wang, X., Feng, F., He, X., Chen, L., Lian, J., & Xie, X. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462862 · Cited by 1,766
Alsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., & McDermott, M. (2019) · Proceedings of the 2nd Clinical Natural Language Processing Workshop · doi:10.18653/v1/w19-1909 · Cited by 1,736
From Word Embeddings To Document Distances
Kusner, M. J., Sun, Y., Kolkin, N., & Weinberger, K. Q. (2015) · PolyPublie (École Polytechnique de Montréal) · Cited by 1,695
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly
Xian, Y., Lampert, C. H., Schiele, B., & Akata, Z. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2857768 · Cited by 1,678
Collaborative Deep Learning for Recommender Systems
Wang, H., Wang, N., & Yeung, D.-Y. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783273 · Cited by 1,657
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Cao, S., Lu, W., & Xu, Q. (2015) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2806416.2806512 · Cited by 1,609
Incorporating Nesterov Momentum into Adam
Dozat, T. (2016) · Cited by 1,595
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591
Neural Word Embedding as Implicit Matrix Factorization
Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,581
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification
Tang, D., Qin, B., & Liu, T. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1167 · Cited by 1,563
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation
Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., & Specia, L. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2001 · Cited by 1,562
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
A primer in BERTology: What we know about how BERT works
Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552
Metadata
- Authors
- Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2013
- Volume
- 26
- Pages
- 3111-3119
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1310.4546
- OpenAlex
- W2153579005
- Cited by
- 24,533
- References
- 19
- Citations by year
- 2026: 114 · 2025: 872 · 2024: 1388 · 2023: 1965 · 2022: 2239 · 2021: 3514 · 2020: 3787 · 2019: 3870 · 2018: 2785 · 2017: 1949 · 2016: 1194 · 2015: 658 · 2014: 176 · 2013: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- distributed word representations, skip-gram model, phrase representation, negative sampling, frequent terms, idioms, word order, training speed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
