The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Improving Distributional Similarity with Lessons Learned from Word Embeddings
Open accessAbstract
Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks. We reveal that much of the performance gains of word embeddings are due to certain system design choices and hyperparameter optimizations, rather than the embedding algorithms themselves. Furthermore, we show that these modifications can be transferred to traditional distributional models, yielding similar gains. In contrast to prior reports, we observe mostly local or insignificant performance differences between the methods, with no global advantage to any single approach over the others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
The Approximation of One Matrix by Another of Lower Rank
Eckart, C., & Young, G. (1936) · Psychometrika · doi:10.1007/bf02288367 · Cited by 3,800
Word association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
Measuring praise and criticism: Inference of semantic orientation from association
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,502
Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1023 · Cited by 1,364
word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
Goldberg, Y., & Levy, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.3722 · Cited by 1,245
Self-citationSimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation
Hill, F., Reichart, R., & Korhonen, A. (2015) · Computational Linguistics · doi:10.1162/coli_a_00237 · Cited by 1,175
Placing search in context: the concept revisited.
Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., & Ruppin, E. (2002) · Cited by 961
A study on similarity and relatedness using distributional and WordNet-based approaches
Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., Paşca, M., & Soroa, A. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics on - NAACL '09 · doi:10.3115/1620754.1620758 · Cited by 860
Extracting semantic representations from word co-occurrence statistics: A computational study
Bullinaria, J. A., & Levy, J. P. (2007) · Behavior Research Methods · doi:10.3758/bf03193020 · Cited by 805
Better Word Representations with Recursive Neural Networks for Morphology
Luong, M.-T., Socher, R., & Manning, C. D. (2013) · Conference on Computational Natural Language Learning · Cited by 786
Dependency-Based Construction of Semantic Space Models
Padó, S., & Lapata, M. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.2.161 · Cited by 695
Distributional Memory: A General Framework for Corpus-Based Semantics
Baroni, M., & Lenci, A. (2010) · Computational Linguistics · doi:10.1162/coli_a_00016 · Cited by 659
Linguistic Regularities in Sparse and Explicit Word Representations
Levy, O., & Goldberg, Y. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1618 · Cited by 628
Self-citationDiscovering word senses from text
Pantel, P., & Lin, D. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775138 · Cited by 591
A word at a time: computing word relatedness using temporal semantic analysis
Radinsky, K., Agichtein, E., Gabrilovich, E., & Markovitch, S. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963455 · Cited by 405
Distributional Semantics in Technicolor
Bruni, E., Boleda, G., Baroni, M., & Tran, N. K. (2012) · Iris (University of Trento) · Cited by 332
Bullinaria, J. A., & Levy, J. P. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0183-8 · Cited by 303
Using wiktionary for computing semantic relatedness
Zesch, T., Müller, C., & Gurevych, I. (2008) · TUbilio (Technical University of Darmstadt) · Cited by 150
Domain and function: A dual-space model of semantic relations and compositions
Turney, P. D. (2012) · Cited by 138
Cited by
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11164 · Cited by 2,312
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications
Cai, H., Zheng, V. W., & Chang, K. C.-C. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2807452 · Cited by 2,034
Comparing different supervised machine learning algorithms for disease prediction
Uddin, S., Khan, A., Hossain, M. E., & Moni, M. A. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-1004-8 · Cited by 1,627
Deep Neural Networks for Learning Graph Representations
Cao, S., Lu, W., & Xu, Q. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10179 · Cited by 1,014
Metadata
- Authors
- Levy, O., Goldberg, Y., & Dagan, I.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2015-12
- Volume
- 3
- Pages
- 211-225
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W1615991656
- Cited by
- 1,308
- References
- 27
- FWCI
- 164.23
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.