The DOI matches its Crossref record; 166 of the 166 references it cites were found in a registry or an index.
From Frequency to Meaning: Vector Space Models of Semantics
Open accessAbstract
Computers understand very little of the meaning of human language. This profoundly limits our ability to give instructions to computers, the ability of computers to explain their actions to us, and the ability of computers to analyse and process text. Vector space models (VSMs) of semantics are beginning to address these limits. This paper surveys the use of VSMs for semantic processing of text. We organize the literature on VSMs according to the structure of the matrix in a VSM. There are currently three broad classes of VSMs, based on term-document, word-context, and pair-pattern matrices, yielding three classes of applications. We survey a broad range of applications in these three categories and we take a detailed look at a specific open source project in each category. Our goal in this survey is to show the breadth of applications of VSMs for semantics, to provide a new perspective on VSMs for those who are already familiar with the area, and to provide pointers into the literature for those who are less familiar with the field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
Jain, A. K., Murty, M. N., & Flynn, P. J. (1999) · ACM Computing Surveys · doi:10.1145/331499.331504 · Cited by 13,199
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Women, Fire, and Dangerous Things
Lakoff, G. (1987) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226471013.001.0001 · Cited by 12,386
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Cited by
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
“to statistical language modeling with considerable success [1]. The follow up work includes applications to automatic speech recognition and machine translation [14, 7], and a wide range of NLP tasks [2, 20, 15, 3, 18, 19, 9]. Recently, Mikolov et al. [8] introduced the Skip-gram model, an efficient method for learning…”
MethodEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
(2010) · Cited by 1,906
Natural speech reveals the semantic maps that tile human cerebral cortex
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016) · Nature · doi:10.1038/nature17637 · Cited by 1,504
Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1023 · Cited by 1,364
Improving Distributional Similarity with Lessons Learned from Word Embeddings
Levy, O., Goldberg, Y., & Dagan, I. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00134 · Cited by 1,308
Recent Trends in Deep Learning Based Natural Language Processing
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02709 · Cited by 1,192
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation
Hill, F., Reichart, R., & Korhonen, A. (2015) · Computational Linguistics · doi:10.1162/coli_a_00237 · Cited by 1,175
Dependency-Based Word Embeddings
Levy, O., & Goldberg, Y. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.3115/v1/p14-2050 · Cited by 1,047
Zero-Shot Learning Through Cross-Modal Transfer
Socher, R., Ganjoo, M., Sridhar, H., Bastani, O., Manning, C. D., & Ng, A. Y. (2013) · arXiv · doi:10.48550/arxiv.1301.3666 · Cited by 997
Composition in Distributional Models of Semantics
Mitchell, J., & Lapata, M. (2010) · Cognitive Science · doi:10.1111/j.1551-6709.2010.01106.x · Cited by 960
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
Hamilton, W. L., Leskovec, J., & Jurafsky, D. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1141 · Cited by 943
Multimodal Distributional Semantics
Bruni, E., Tran, N. K., & Baroni, M. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4135 · Cited by 908
How to do Linguistics with R: Data exploration and statistical analysis
Levshina, N. (2015) · John Benjamins Publishing Company eBooks · doi:10.1075/z.195 · Cited by 862
Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology
Maier, D., Waldherr, A., Miltner, P., Wiedemann, G., Niekler, A., Keinert, A., Pfetsch, B., Heyer, G., Reber, U., Häussler, T., et al. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2018.1430754 · Cited by 832
Neural Network Methods for Natural Language Processing
Goldberg, Y. (2017) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02165-7 · Cited by 821
Aletras, N., Tsarapatsanis, D., Preoţiuc-Pietro, D., & Lampos, V. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.93 · Cited by 810
Grounded Compositional Semantics for Finding and Describing Images with Sentences
Socher, R., Karpathy, A., Le, Q. V., Manning, C. D., & Ng, A. Y. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00177 · Cited by 777
A Model of Text for Experimentation in the Social Sciences
Roberts, M. E., Stewart, B. M., & Airoldi, E. M. (2016) · Journal of the American Statistical Association · doi:10.1080/01621459.2016.1141684 · Cited by 728
Distributional Memory: A General Framework for Corpus-Based Semantics
Baroni, M., & Lenci, A. (2010) · Computational Linguistics · doi:10.1162/coli_a_00016 · Cited by 659
Linguistic Regularities in Sparse and Explicit Word Representations
Levy, O., & Goldberg, Y. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1618 · Cited by 628
Topic Modeling in Management Research: Rendering New Theory from Textual Data
Hannigan, T. R., Haans, R. F. J., Vakili, K., Tchalian, H., Glaser, V. L., Wang, M. S., Kaplan, S., & Jennings, P. D. (2019) · Academy of Management Annals · doi:10.5465/annals.2017.0099 · Cited by 621
Improving Vector Space Word Representations Using Multilingual Correlation
Faruqui, M., & Dyer, C. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1049 · Cited by 620
Grounding Action Descriptions in Videos
Regneri, M., Rohrbach, M., Wetzel, D., Thater, S., Schiele, B., & Pinkal, M. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00207 · Cited by 467
Distributional Models of Word Meaning
Lenci, A. (2018) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-030514-125254 · Cited by 447
Contextual semantics for sentiment analysis of Twitter
Saif, H., He, Y., Fernandez, M., & Alani, H. (2016) · Information Processing & Management · doi:10.1016/j.ipm.2015.01.005 · Cited by 425
Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification
Zhuang, C., & Ma, Q. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186116 · Cited by 423
Shapiro, A. H., Sudhof, M., & Wilson, D. J. (2022) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.07.053 · Cited by 423
Big Data: Deep Learning for financial sentiment analysis
Sohangir, S., Wang, D., Pomeranets, A., & Khoshgoftaar, T. M. (2018) · Journal of Big Data · doi:10.1186/s40537-017-0111-6 · Cited by 390
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, É., Joulin, A., & Mikolov, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04606 · Cited by 384
Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings
Song, Y., Shi, S., Li, J., & Zhang, H. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2028 · Cited by 380
From Word To Sense Embeddings: A Survey on Vector Representations of Meaning
Camacho-Collados, J., & Pilehvar, M. T. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11259 · Cited by 363
Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora
Hamilton, W. L., Clark, K., Leskovec, J., & Jurafsky, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1057 · Cited by 355
Evaluation of Sentiment Analysis in Finance: From Lexicons to Transformers
Mishev, K., Gjorgjevikj, A., Vodenska, I., Chitkushev, L. T., & Trajanov, D. (2020) · IEEE Access · doi:10.1109/access.2020.3009626 · Cited by 345
Günther, F., Rinaldi, L., & Marelli, M. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619861372 · Cited by 341
Vector Space Models of Word Meaning and Phrase Meaning: A Survey
Erk, K. (2012) · Language and Linguistics Compass · doi:10.1002/lnco.362 · Cited by 321
Co-LSTM: Convolutional LSTM model for sentiment analysis in social big data
Behera, R. K., Jena, M., Rath, S. K., & Misra, S. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2020.102435 · Cited by 320
Schmiedel, T., Müller, O., & vom Brocke, J. (2019) · Organizational Research Methods · doi:10.1177/1094428118773858 · Cited by 319
An Introduction to Neural Information Retrieval
Mitra, B., & Craswell, N. (2018) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000061 · Cited by 317
Semantic memory: A review of methods, models, and current challenges
Kumar, A. A. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01792-x · Cited by 314
Gaussian LDA for Topic Models with Word Embeddings
Das, R., Zaheer, M., & Dyer, C. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1077 · Cited by 313
Springer Handbook of Science and Technology Indicators
(2019) · Springer Handbooks · doi:10.1007/978-3-030-02511-3 · Cited by 300
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291
A Latent Variable Model Approach to PMI-based Word Embeddings
Arora, S., Li, Y., Liang, Y., Ma, T., & Risteski, A. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00106 · Cited by 286
Utilizing big data analytics for information systems research: challenges, promises and guidelines
Müller, O., Junglas, I., Brocke, J. V., & Debortoli, S. (2016) · European Journal of Information Systems · doi:10.1057/ejis.2016.2 · Cited by 281
Combining Language and Vision with a Multimodal Skip-gram Model
Lazaridou, A., Pham, N. T., & Baroni, M. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1016 · Cited by 280
The Hierarchical Cortical Organization of Human Speech Processing
de Heer, W. A., Huth, A. G., Griffiths, T. L., Gallant, J. L., & Theunissen, F. E. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.3267-16.2017 · Cited by 277
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 274
Topic Modeling: A Comprehensive Review
Kherwa, P., & Bansal, P. (2018) · ICST Transactions on Scalable Information Systems · doi:10.4108/eai.13-7-2018.159623 · Cited by 264
NeuroQuery, comprehensive meta-analysis of human brain mapping
Dockès, J., Poldrack, R. A., Primet, R., Gözükan, H., Yarkoni, T., Suchanek, F., Thirion, B., & Varoquaux, G. (2020) · eLife · doi:10.7554/elife.53385 · Cited by 258
Semantics-Aware Content-Based Recommender Systems
de Gemmis, M., Lops, P., Musto, C., Narducci, F., & Semeraro, G. (2015) · Recommender Systems Handbook · doi:10.1007/978-1-4899-7637-6_4 · Cited by 258
New Horizons for a Data-Driven Economy
(2016) · Springer eBooks · doi:10.1007/978-3-319-21569-3 · Cited by 253
Deniz, F., Nunez-Elizalde, A. O., Huth, A. G., & Gallant, J. L. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.0675-19.2019 · Cited by 214
“The semantic content of the stories was estimated continuously by projecting the narrative into a word embedding space based on word co-occurrence statistics (Church and Hanks, 1990; Lund and Burgess, 1996; Mitchell et al., 2008; Turney and Pantel, 2010; Wehbe et al., 2014).”
MethodText mining for information systems researchers: An annotated topic modeling tutorial
University of Liechtenstein, Debortoli, S., Müller, O., IT University of Copenhagen, Junglas, I., Florida State University, vom Brocke, J., & University of Liechtenstein (2016) · Communications of the Association for Information Systems · doi:10.17705/1cais.03907 · Cited by 199
Compositionality Decomposed: How do Neural Networks Generalise?
Hupkes, D., Dankers, V., Mul, M., & Bruni, E. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11674 · Cited by 194
Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Arora, S., Li, Y., Liang, Y., Ma, T., & Risteski, A. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00034 · Cited by 186
A Survey of Paraphrasing and Textual Entailment Methods
Androutsopoulos, I., & Malakasiotis, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2985 · Cited by 178
Hasson, U., Egidi, G., Marelli, M., & Willems, R. M. (2018) · Cognition · doi:10.1016/j.cognition.2018.06.018 · Cited by 157
Tran, T., Nguyen, T. D., Phung, D., & Venkatesh, S. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.01.012 · Cited by 172
Altszyler, E., Ribeiro, S., Sigman, M., & Fernández Slezak, D. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.09.004 · Cited by 128
“The success of LSA in capturing the latent meaning of words comes from this low-dimensional mapping (Turney & Pantel, 2010).”
BackgroundA Timely Intervention: Tracking the Changing Meanings of Political Concepts with Word Vectors
Rodman, E. (2020) · Political Analysis · doi:10.1017/pan.2019.23 · Cited by 112
HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Vulić, I., Gerz, D., Kiela, D., Hill, F., & Korhonen, A. (2016) · arXiv · doi:10.48550/arxiv.1608.02117 · Cited by 90
Does ChatGPT have semantic understanding? A problem with the statistics-of-occurrence strategy
Titus, L. M. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101174 · Cited by 82
Distributional vectors encode referential attributes
Gupta, A., Boleda, G., Baroni, M., & Padó, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1002 · Cited by 79
Anderson, A. J., Bruni, E., Lopopolo, A., Poesio, M., & Baroni, M. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.06.093 · Cited by 77
Burkhardt, H. A., Alexopoulos, G. S., Pullmann, M. D., Hull, T. D., Areán, P. A., & Cohen, T. (2021) · Journal of Medical Internet Research · doi:10.2196/28244 · Cited by 68
Conceptualizing generative AI as style engines: Application archetypes and implications
Riemer, K., & Peter, S. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2024.102824 · Cited by 63
Automatic Poetry Generation from Prosaic Text
Van de Cruys, T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.223 · Cited by 50
Exploring Numeracy in Word Embeddings
Naik, A., Ravichander, A., Rose, C., & Hovy, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1329 · Cited by 49
Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space
Nastase, S. A., Liu, Y.-F., Hillman, H., Norman, K. A., & Hasson, U. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116865 · Cited by 42
Concepts in the Brain: The View From Cross-linguistic Diversity
Kemmerer, D. (2019) · doi:10.1093/oso/9780190682620.001.0001 · Cited by 42
Ewbank, M. P., Cummins, R., Tablan, V., Catarino, A., Buchholz, S., & Blackwell, A. D. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1788740 · Cited by 39
Newman-Griffis, D., Camacho Maldonado, J., Ho, P.-S., Sacco, M., Jimenez Silva, R., Porcino, J., & Chan, L. (2021) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2021.742702 · Cited by 27
TwitchChat: A Dataset for Exploring Livestream Chat
Ringer, C., Nicolaou, M., & Walker, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v16i1.7439 · Cited by 11
Allman, B. E., Vieira, L., Diaz, D. J., & Wilke, C. O. (2025) · Journal of The Royal Society Interface · doi:10.1098/rsif.2024.0598 · Cited by 11
Identification, Interpretability, and
Lauretig, A. (2019) · Proceedings of the Third Workshop on Natural Language Processing and Computational Social Science · doi:10.18653/v1/w19-2102 · Cited by 9
Overconfidence without Understanding: AI Explanations Increase the Illusion of Explanatory Depth
Aslanov, I., Felmer, P., & Guerra, E. (2025) · doi:10.31234/osf.io/8psgf_v1
Metadata
- Authors
- Turney, P. D., & Pantel, P.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2010-02-27
- Volume
- 37
- Pages
- 141-188
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W1662133657
- Cited by
- 2,866
- References
- 166
- FWCI
- 119.37
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.