The DOI matches its Crossref record; 64 of the 66 references it cites were found in a registry or an index.
A survey of word embeddings based on deep learning
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Specifically, of the 15 tools dedicated to classifying relevant papers, the majority (8 out of 15) use a Bag of Words (BoW) approach (Zhang et al. 2010), while the remainder employ various word embedding techniques (Wang et al. 2020).”
From Artificial intelligence for literature reviews: opportunities and challenges (2024) · cites this paper as Wang et al. 2020
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“…and the relationship between target word and its context words through neural networks and contain fruitful semantic information in the embeddings, which can be obtained by training language models [19, 20, 22–27], or by building specifically neural networks for generating word embeddings [28–36].”
BackgroundA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
(1992) · doi:10.5555/176313.176316 · Cited by 2,892
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 1×
“[69] proposed InferSent that used Bi-LSTM as sentence encoder and proved their model performed better on natural language inference task.”
Method(2010) · Cited by 1,906
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10362 · Cited by 1,491
Understanding bag-of-words model: a statistical framework
Zhang, Y., Jin, R., & Zhou, Z.-H. (2010) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-010-0001-0 · Cited by 1,384
Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1023 · Cited by 1,364
(2011) · Cited by 1,192
Distributional clustering of English words
Pereira, F., Tishby, N., & Lee, L. (1993) · Proceedings of the 31st annual meeting on Association for Computational Linguistics - · doi:10.3115/981574.981598 · Cited by 974
Vector-based Models of Semantic Composition
Mitchell, J., & Lapata, M. (2008) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 872
A Scalable Hierarchical Distributed Language Model
Mnih, A., & Hinton, G. E. (2008) · Cited by 838
(2005) · Cited by 823
Better Word Representations with Recursive Neural Networks for Morphology
Luong, M.-T., Socher, R., & Manning, C. D. (2013) · Conference on Computational Natural Language Learning · Cited by 786
Linguistic Regularities in Sparse and Explicit Word Representations
Levy, O., & Goldberg, Y. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1618 · Cited by 628
Three new graphical models for statistical language modelling
Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273577 · Cited by 563
Similarity-Based Models of Word Cooccurrence Probabilities
Dagan, I., Lee, L., & Pereira, F. C. N. (1999) · Machine Learning · doi:10.1023/a:1007537716579 · Cited by 375
Joint learning of character and word embeddings
Chen, X., Xu, L., Liu, Z., Sun, M., & Luan, H. (2015) · International Conference on Artificial Intelligence · Cited by 262
2010 International Conference on Artificial Intelligence
Yang, Y., Guglielmino, E., Dai, J. S., Boaventura, T., & Caldwell, D. G. (2010) · Cited by 253
Multi-View Learning of Word Embeddings via CCA
Dhillon, P. S., Foster, D. P., & Ungar, L. (2011) · ScholarlyCommons (University of Pennsylvania) · Cited by 241
Phrase clustering for discriminative learning
Lin, D., & Wu, X. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690290 · Cited by 226
Word Embeddings through Hellinger PCA
Lebret, R., & Collobert, R. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1051 · Cited by 214
Compositional Morphology for Word Representations and Language ModellingPreprint
Botha, J. A., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4273 · Cited by 211
Sanderson, M. (2010) · Natural Language Engineering · doi:10.1017/s1351324909005129 · Cited by 209
BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages
Heinzerling, B., & Strube, M. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3eqtu58tsqay · Cited by 181
Recurrent Convolutional Neural Networks for Discourse Compositionality
Kalchbrenner, N., & Blunsom, P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.3584 · Cited by 178
Knowledge-Powered Deep Learning for Word Embedding
Bian, J., Gao, B., & Liu, T.-Y. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-662-44848-9_9 · Cited by 161
Schütze, H. (2003) · ACM/IEEE Conference on Supercomputing (SC) · doi:10.1109/superc.1992.236684 · Cited by 155
cw2vec: Learning Chinese Word Embeddings with Stroke n-gram Information
Cao, S., Lu, W., Zhou, J. W., & Li, X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12029 · Cited by 138
Contextual word similarity and estimation from sparse data
Dagan, I., Marcus, S., & Markovitch, S. (1993) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981574.981596 · Cited by 124
Joint Embeddings of Chinese Words, Characters, and Fine-grained Subcharacter Components
Yu, J., Jian, X., Xin, H., & Song, Y. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1027 · Cited by 121
Radical-Enhanced Chinese Character Embedding
Sun, Y., Lin, L., Yang, N., Ji, Z., & Wang, X. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-12640-1_34 · Cited by 120
Dict2vec : Learning Word Embeddings using Lexical Dictionaries
Tissier, J., Gravier, C., & Habrard, A. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1024 · Cited by 107
Cotterell, R., & Schütze, H. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1140 · Cited by 104
Component-Enhanced Chinese Character Embeddings
Li, Y., Li, W., Sun, F., & Li, S. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1098 · Cited by 101
WCP-RNN: a novel RNN-based approach for Bio-NER in Chinese EMRs
Li, J., Zhao, S., Yang, J., Huang, Z., Лю, Б., Chen, S., Pan, H., & Wang, Q. (2018) · The Journal of Supercomputing · doi:10.1007/s11227-017-2229-x · Cited by 101
A Joint Model for Word Embedding and Word Morphology
Cao, K., & Rei, M. (2016) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w16-1603 · Cited by 98
Can artificial neural networks learn language models?
Xu, W. H., & Rudnicky, A. I. (2000) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2000-50 · Cited by 95
Improve Chinese Word Embeddings by Exploiting Internal Structure
Xu, J. L., Liu, J., Zhang, L., Li, Z., & Chen, H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1119 · Cited by 85
Eigenwords: spectral word embeddings
Dhillon, P. S., Foster, D. P., & Ungar, L. (2015) · Journal of Machine Learning Research · Cited by 74
Chinese Named Entity Recognition with Character-Word Mixed Embedding
Shijia, E., & Xiang, Y. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133088 · Cited by 28
International Conference on Learning Representations Workshop track
Yousefi, F., Dai, Z., Ek, C. H., & Lawrence, N. D. (2016) · Cited by 25
Mining coherent topics in documents using word embeddings and large-scale text data
Yao, L., Zhang, Y., Chen, Q., Qian, H., Wei, B., & Hu, Z. (2017) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2017.06.024 · Cited by 23
Can deep learning techniques improve classification performance of vandalism detection in Wikipedia?
Martinez-Rico, J. R., Martínez-Romo, J., & Araujo, L. (2018) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2018.11.012 · Cited by 22
Empirical study on character level neural network classifier for Chinese text
Chung, T., Xu, B., Liu, Y., Ouyang, C., Li, S., & Luo, L. (2019) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2019.01.009 · Cited by 21
Improved Learning of Chinese Word Embeddings with Semantic Knowledge
Yang, L., & Sun, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25816-4_2 · Cited by 9
Research works printed but not linked: 14
- Firth JR (1957) A synopsis of linguistic theory, 1930–1955. In: Studies in linguistic analysis, Philological Society, Oxford
- Schütze H (1992) Context space. In: AAAI fall symposium on probabilistic approaches to natural language, pp 113–120
- Bengio Y, Ducharme R, Vincent P, Jauvin C (2003) A neural probabilistic language model. J Mach Learn Res 3:1137–1155
- Turian J, Ratinov L, Bengio Y (2010) Word representations: a simple and general method for semi-supervised learning. In: Proceedings of the 48th annual meeting of the association for computational linguistics (ACL), pp 384–394
- Mnih A, Kavukcuoglu K (2013) Learning word embeddings efficiently with noise-contrastive estimation. In: Advances in neural information processing systems, pp 2265–2273
- Xu Y, Liu J (2017) Implicitly incorporating morphological information into word embedding. arXiv preprint arXiv:170102481
- Talman A, Yli-Jyra A, Tiedemann J (2018) Natural language inference with hierarchical Bilstm max pooling architecture. arXiv preprint arXiv:180808762
- Sennrich R, Haddow B, Birch A (2015) Neural machine translation of rare words with subword units. arXiv preprint arXiv:150807909
- Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
- Blacoe W, Lapata M (2012) A comparison of vector-based representations for semantic composition. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, Association for Computational Linguistics, pp 546–556
- Kiros R, Zhu Y, Salakhutdinov RR, Zemel R, Urtasun R, Torralba A, Fidler S (2015) Skip-thought vectors. In: Advances in neural information processing systems, pp 3294–3302
- Logeswaran L, Lee H (2018) An efficient framework for learning sentence representations. arXiv preprint arXiv:1803.02893 · arXiv
- Conneau A, Kiela D, Schwenk H et al (2017) Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364 · DOI · Paper page
- Su TR, Lee HY (2017) Learning chinese word representations from glyphs of characters. arXiv preprint arXiv:170804755
Cited by
Impact of word embedding models on text analytics in deep learning environment: a review
Asudani, D. S., Nagwani, N. K., & Singh, P. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10419-1 · Cited by 274
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246
“Text representation - describes the processes employed to convert text into suitable input for the algorithm (e.g., BoW [76], LDA topics [77], word embeddings [78]).”
Transformer-based deep learning models for the sentiment analysis of social media data
Tabinda Kokab, S., Asghar, S., & Naz, S. (2022) · Array · doi:10.1016/j.array.2022.100157 · Cited by 237
Khan, W., Daud, A., Khan, K., Muhammad, S., & Haq, R. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100026 · Cited by 217
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT
Ganesh, P., Chen, Y., Lou, X., Khan, M. A., Yang, Y., Sajjad, H., Nakov, P., Chen, D., & Winslett, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00413 · Cited by 190
Metadata
- Authors
- Wang, S., Zhou, W., & Jiang, C.
- Affiliations
- Beijing University of Posts and Telecommunications (CN)
- Corresponding author
- Zhou, Wenan
- Venue
- Computing
- Published
- 2019-11-12
- Volume
- 102
- Issue
- 3
- Pages
- 717-740
- ISSN
- 0010-485X (ISSN-L)
- 1436-5057
- OpenAlex
- W2988533489
- Cited by
- 193
- References
- 66
- Citations by year
- 2026: 4 · 2025: 40 · 2024: 45 · 2023: 40 · 2022: 31 · 2021: 24 · 2020: 9
- FWCI
- 7.38
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Computing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- word embeddings, word meaning, multi-source data, knowledge representation, text classification, question answering, knowledge mining
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.