The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors
Open accessAbstract
Context-predicting models (more commonly known as embeddings or neural language models) are the new kids on the distributional semantics block.Despite the buzz surrounding these models, the literature is still lacking a systematic comparison of the predictive models with classic, count-vector-based distributional semantic approaches.In this paper, we perform such an extensive evaluation, on a wide range of lexical semantics tasks and across many parameter settings.The results, to our own surprise, show that the buzz is fully justified, as the context-predicting models obtain a thorough and resounding victory against their count-based counterparts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
ALGORITHMS FOR NON-NEGATIVE MATRIX FACTORIZATION
Lee, D. D., & Seung, H. S. (2000) · neural information processing systems · Cited by 9,873
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Golub, G. H., & Loan, C. F. V. (1996) · Johns Hopkins University Press eBooks · Cited by 6,027
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Cited by
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Recurrent Convolutional Neural Networks for Text Classification
Lai, S., Xu, L., Liu, K., & Zhao, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9513 · Cited by 2,280
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification
Tang, D., Qin, B., & Liu, T. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1167 · Cited by 1,519
Improving Distributional Similarity with Lessons Learned from Word Embeddings
Levy, O., Goldberg, Y., & Dagan, I. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00134 · Cited by 1,308
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation
Hill, F., Reichart, R., & Korhonen, A. (2015) · Computational Linguistics · doi:10.1162/coli_a_00237 · Cited by 1,175
Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks
Chen, Y., Xu, L., Liu, K., Zeng, D., & Zhao, J. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1017 · Cited by 906
Joint Event Extraction via Recurrent Neural Networks
Nguyen, T. H., Cho, K., & Grishman, R. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1034 · Cited by 647
Linguistic Regularities in Sparse and Explicit Word Representations
Levy, O., & Goldberg, Y. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1618 · Cited by 628
Learning Distributed Representations of Sentences from Unlabelled Data
Hill, F., Cho, K., & Korhonen, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1162 · Cited by 566
Evaluation methods for unsupervised word embeddings
Schnabel, T., Labutov, I., Mimno, D., & Joachims, T. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1036 · Cited by 540
Identifying Customer Needs from User-Generated Content
Timoshenko, A., & Hauser, J. R. (2019) · Marketing Science · doi:10.1287/mksc.2018.1123 · Cited by 513
SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability
Agirre, E., Banea, C., Cardie, C., Cer, D., Diab, M., Gonzalez-Agirre, A., Guo, W., Lopez-Gazpio, I., Maritxalar, M., Mihalcea, R., et al. (2015) · Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015) · doi:10.18653/v1/s15-2045 · Cited by 466
Short Text Similarity with Word Embeddings
Kenter, T., & de Rijke, M. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806475 · Cited by 440
Mandera, P., Keuleers, E., & Brysbaert, M. (2017) · Journal of Memory and Language · doi:10.1016/j.jml.2016.04.001 · Cited by 428
Broderick, M. P., Anderson, A. J., Di Liberto, G. M., Crosse, M. J., & Lalor, E. C. (2018) · Current Biology · doi:10.1016/j.cub.2018.01.080 · Cited by 421
Liu, X., Gao, J., He, X., Deng, L., Duh, K., & Wang, Y.-Y. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1092 · Cited by 395
Liu, Y., Liu, Z., Chua, T.-S., & Sun, M. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9522 · Cited by 381
A comparison of word embeddings for the biomedical natural language processing
Wang, Y., Liu, S., Afzal, N., Rastegar-Mojarad, M., Wang, L., Shen, F., Kingsbury, P., & Liu, H. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.09.008 · Cited by 370
Toward a universal decoder of linguistic meaning from brain activation
Pereira, F., Lou, B., Pritchett, B., Ritter, S., Gershman, S. J., Kanwisher, N., Botvinick, M., & Fedorenko, E. (2018) · Nature Communications · doi:10.1038/s41467-018-03068-4 · Cited by 324
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
Qin, G., & Eisner, J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.410 · Cited by 342
Günther, F., Rinaldi, L., & Marelli, M. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619861372 · Cited by 341
Sentiment Embeddings with Applications to Sentiment Analysis
Tang, D., Wei, F., Qin, B., Yang, N., Liu, T., & Zhou, M. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2489653 · Cited by 335
Automating creativity assessment with SemDis: An open platform for computing semantic distance
Beaty, R. E., & Johnson, D. R. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01453-w · Cited by 341
“This space also shows robust associations with human judgements of relatedness and was shown to be the best-performing model compared to multiple CBOW and LSA-based count models (Baroni, Dinu, & Kruszewski, 2014).”
MethodEvaluating word embedding models: methods and experimental results
Wang, B., Wang, A., Chen, F., Wang, Y., & Kuo, C.-C. J. (2019) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2019.12 · Cited by 323
A recent overview of the state-of-the-art elements of text classification
Mirończuk, M. M., & Protasiewicz, J. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.058 · Cited by 317
Semantic memory: A review of methods, models, and current challenges
Kumar, A. A. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01792-x · Cited by 314
How to Generate a Good Word Embedding
Lai, S., Liu, K., He, S., & Zhao, J. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.45 · Cited by 312
Improving zero-shot learning by mitigating the hubness problem
Dinu, G., Lazaridou, A., & Baroni, M. (2015) · Figshare · doi:10.5281/zenodo.2654863 · Cited by 310
Self-citationEvolution of Semantic Similarity—A Survey
Chandrasekaran, D., & Mago, V. (2022) · ACM Computing Surveys · doi:10.1145/3440755 · Cited by 303
Monolingual and Cross-Lingual Information Retrieval Models Based on (Bilingual) Word Embeddings
Vulić, I., & Moens, M.-F. (2015) · Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2766462.2767752 · Cited by 294
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04207 · Cited by 283
Ye, X., Shen, H., Ma, X., Bunescu, R., & Liu, C. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884862 · Cited by 282
Combining Language and Vision with a Multimodal Skip-gram Model
Lazaridou, A., Pham, N. T., & Baroni, M. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1016 · Cited by 280
Self-citationArabic text classification using deep learning models
Elnagar, A., Al-Debsi, R., & Einea, O. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2019.102121 · Cited by 274
Word Representations via Gaussian Embedding
Vilnis, L., & McCallum, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6623 · Cited by 267
SensEmbed: Learning Sense Embeddings for Word and Relational Similarity
Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1010 · Cited by 267
Word Embeddings: What Works, What Doesn’t, and How to Tell the Difference for Applied Research
Rodriguez, P. L., & Spirling, A. (2022) · The Journal of Politics · doi:10.1086/715162 · Cited by 254
Embeddings for Word Sense Disambiguation: An Evaluation Study
Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1085 · Cited by 253
A Survey on Text Classification Algorithms: From Text to Predictions
Gasparetto, A., Marcuzzo, M., Zangari, A., & Albarelli, A. (2022) · Information · doi:10.3390/info13020083 · Cited by 253
Academic Plagiarism Detection: A Systematic Literature Review
Foltýnek, T., Meuschke, N., & Gipp, B. (2020) · ACM Computing Surveys · doi:10.1145/3345317 · Cited by 238
A survey of word embeddings based on deep learning
Wang, S., Zhou, W., & Jiang, C. (2020) · Computing · doi:10.1007/s00607-019-00768-7 · Cited by 193
Word meaning in minds and machines.
Lake, B. M., & Murphy, G. L. (2023) · Psychological Review · doi:10.1037/rev0000297 · Cited by 131
Altszyler, E., Ribeiro, S., Sigman, M., & Fernández Slezak, D. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.09.004 · Cited by 128
SECNLP: A survey of embeddings in clinical natural language processing
Kalyan, K. S., & Sangeetha, S. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103323 · Cited by 111
Beaty, R. E., Zeitlen, D. C., Baker, B. S., & Kenett, Y. N. (2021) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2021.100859 · Cited by 86
HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Vulić, I., Gerz, D., Kiela, D., Hill, F., & Korhonen, A. (2016) · arXiv · doi:10.48550/arxiv.1608.02117 · Cited by 90
Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets
Dror, R., Baumer, G., Bogomolov, M., & Reichart, R. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00074 · Cited by 63
Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans
Digutsch, J., & Kosinski, M. (2023) · Scientific Reports · doi:10.1038/s41598-023-32248-6 · Cited by 49
Towards Trustworthy AutoGrading of Short, Multi-lingual, Multi-type Answers
Schneider, J., Richner, R., & Riser, M. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00289-z · Cited by 48
Ewbank, M. P., Cummins, R., Tablan, V., Catarino, A., Buchholz, S., & Blackwell, A. D. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1788740 · Cited by 39
Context-aware Path Ranking for Knowledge Base Completion
Mazumder, S., & Liu, B. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/166 · Cited by 35
Analysis of sentence embedding models using prediction tasks in natural language processing
Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., & Goldberg, Y. (2017) · IBM Journal of Research and Development · doi:10.1147/jrd.2017.2702858 · Cited by 21
A structured distributional model of sentence meaning and processing
Chersoni, E., Santus, E., Pannitto, L., Lenci, A., Blache, P., & Huang, C.-R. (2019) · Natural Language Engineering · doi:10.1017/s1351324919000214 · Cited by 19
Distributional Semantics: Meaning Through Culture and Interaction
Contreras Kallens, P., & Christiansen, M. H. (2025) · Topics in Cognitive Science · doi:10.1111/tops.12771 · Cited by 17
Semantic embeddings of generic objects for zero-shot learning
Hascoet, T., Ariki, Y., & Takiguchi, T. (2019) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-018-0371-x · Cited by 9
Skills2Graph: Processing million Job Ads to face the Job Skill Mismatch Problem
Giabelli, A., Malandri, L., Mercorio, F., Mezzanzanica, M., & Seveso, A. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/708 · Cited by 7
Metadata
- Authors
- Baroni, M., Dinu, G., & Kruszewski, G.
- Venue
- Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2014
- Pages
- 238-247
- OpenAlex
- W2251803266
- Cited by
- 1,364
- References
- 44
- FWCI
- 96.45
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.