The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Vector Space Models of Word Meaning and Phrase Meaning: A Survey
Abstract
Distributional models represent a word through the contexts in which it has been observed. They can be used to predict similarity in meaning, based on the distributional hypothesis, which states that two words that occur in similar contexts tend to have similar meanings. Distributional approaches are often implemented in vector space models. They represent a word as a point in high‐dimensional space, where each dimension stands for a context item, and a word's coordinates represent its context counts. Occurrence in similar contexts then means proximity in space. In this survey we look at the use of vector space models to describe the meaning of words and phrases: the phenomena that vector space models address, and the techniques that they use to do so. Many word meaning phenomena can be described in terms of semantic similarity: synonymy, priming, categorization, and the typicality of a predicate's arguments. But vector space models can do more than just predict semantic similarity. They are a very flexible tool, because they can make use of all of linear algebra, with all its data structures and operations. The dimensions of a vector space can stand for many things: context words, or non‐linguistic context like images, or properties of a concept. And vector space models can use matrices or higher‐order arrays instead of vectors for representing more complex relationships. Polysemy is a tough problem for distributional approaches, as a representation that is learned from all of a word's contexts will conflate the different senses of the word. It can be addressed, using either clustering or vector combination techniques. Finally, we look at vector space models for phrases, which are usually constructed by combining word vectors. Vector space models for phrases can predict phrase similarity, and some argue that they can form the basis for a general‐purpose representation framework for natural language semantics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Barsalou, L. W. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093639 · Cited by 5,936 · cited 1×
“Theories of embodied cognition propose that concepts are organized in the brain based on their sensory and motor properties, and that simulation of actions and perceptions in the brain plays a central role in cognition (Barsalou 2008).”
BackgroundHarris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
Hall, R., Wittgenstein, L., & Anscombe, G. E. M. (1967) · The Philosophical Quarterly · doi:10.2307/2217461 · Cited by 3,995
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/jmlr.2003.3.4-5.993 · Cited by 3,024
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
The Cambridge handbook of thinking and reasoning
Holyoak, K. J., & Morrison, B. (2005) · Cited by 2,132
Handbook of Latent Semantic Analysis
(2007) · Psychology Press eBooks · doi:10.4324/9780203936399 · Cited by 1,554
Evaluating WordNet-based Measures of Lexical Semantic Relatedness
Budanitsky, A., & Hirst, G. (2006) · Computational Linguistics · doi:10.1162/089120106776173093 · Cited by 1,448
Contextual correlates of synonymy
Rubenstein, H., & Goodenough, J. B. (1965) · Communications of the ACM · doi:10.1145/365628.365657 · Cited by 1,425
Automatic word sense discrimination
Schütze, H. (1998) · doi:10.5555/972719.972724 · Cited by 1,266
Semantic feature production norms for a large set of living and nonliving things
McRae, K., Cree, G. S., Seidenberg, M. S., & Mcnorgan, C. (2005) · Behavior Research Methods · doi:10.3758/bf03192726 · Cited by 1,068
Composition in Distributional Models of Semantics
Mitchell, J., & Lapata, M. (2010) · Cognitive Science · doi:10.1111/j.1551-6709.2010.01106.x · Cited by 960
Smolensky, P. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90007-m · Cited by 959
Studies in linguistic analysis
Firth, J. R. (1957) · Blackwell Publishing Ltd eBooks · Cited by 750
Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension
McRae, K., Spivey-Knowlton, M. J., & Tanenhaus, M. K. (1998) · Journal of Memory and Language · doi:10.1006/jmla.1997.2543 · Cited by 742
Dependency-Based Construction of Semantic Space Models
Padó, S., & Lapata, M. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.2.161 · Cited by 695
Koehn, P., & Hoang, H. T. (2007) · The Association for Computational Linguistics · Cited by 673
Distributional Memory: A General Framework for Corpus-Based Semantics
Baroni, M., & Lenci, A. (2010) · Computational Linguistics · doi:10.1162/coli_a_00016 · Cited by 659
Representing word meaning and order information in a composite holographic lexicon.
Jones, M. N., & Mewhort, D. J. K. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.1 · Cited by 613
Discovery of inference rules for question-answering
LIN, D., & PANTEL, P. (2001) · Natural Language Engineering · doi:10.1017/s1351324901002765 · Cited by 526
"I Don’t Believe in Word Senses"
Kilgarriff, A. (1997) · Computers and the Humanities · doi:10.1023/a:1000583911091 · Cited by 493
Integrating experiential and distributional data to learn semantic representations.
Andrews, M., Vigliocco, G., & Vinson, D. (2009) · Psychological Review · doi:10.1037/a0016261 · Cited by 454
Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. F. (2004) · Cognitive Psychology · doi:10.1016/j.cogpsych.2003.09.001 · Cited by 438
Symbol Interdependency in Symbolic and Embodied Cognition
Louwerse, M. M. (2011) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2010.01106.x · Cited by 399
Kaplan, R. M. (2010) · Cited by 380
A structured vector space model for word meaning in context
Erk, K., & Padó, S. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613831 · Cited by 366
Self-citationSimilarity of Semantic RelationsPreprint
Turney, P. D. (2006) · CogPrints (University of Southampton) · Cited by 359
Multi-Prototype Vector-Space Models of Word Meaning
Reisinger, J., & Mooney, R. J. (2010) · Cited by 347
Modelling Parsing Constraints with High-dimensional Context Space
Lund, C. B. A. K. (1997) · Language and Cognitive Processes · doi:10.1080/016909697386844 · Cited by 314
Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)
Huang, C., & Jurafsky, D. (2010) · International Conference on Computational Linguistics · Cited by 307
Randomized algorithms and NLP: using locality sensitive hash function for high speed noun clustering
Ravichandran, D., Pantel, P., & Hovy, E. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219917 · Cited by 208
Kintsch, W. (2001) · Cognitive Science · doi:10.1207/s15516709cog2502_1 · Cited by 207
Similarity of Semantic Relations
Turney, P. D. (2006) · Computational Linguistics · doi:10.1162/coli.2006.32.3.379 · Cited by 193
Strudel: A Corpus‐Based Semantic Model Based on Properties and Types
Baroni, M., Murphy, B., Barbu, E., & Poesio, M. (2010) · Cognitive Science · doi:10.1111/j.1551-6709.2009.01068.x · Cited by 135
Conceptual Similarity Across Sensory and Neural Diversity
Churchland, P. M. (1998) · The Journal of Philosophy · doi:10.5840/jphil19989514 · Cited by 113
Conceptual Similarity across Sensory and Neural Diversity: The Fodor/Lepore Challenge Answered
Churchland, P. M. (1998) · The Journal of Philosophy · doi:10.2307/2564566 · Cited by 76
Semi-supervised semantic role labeling using the latent words language model
Deschacht, K., & Moens, M. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699510.1699514 · Cited by 72
A Context-Theoretic Framework for Compositionality in Distributional Semantics
Clarke, D. (2011) · Computational Linguistics · doi:10.1162/coli_a_00084 · Cited by 62
Fodor, J. A. (1997) · Cognition · doi:10.1016/s0010-0277(96)00780-9 · Cited by 52
Bos, J. (2011) · Language and Linguistics Compass · doi:10.1111/j.1749-818x.2011.00284.x · Cited by 27
Devereux, B. J., Pilkington, N. C. V., Poibeau, T., & Korhonen, A. (2009) · Research on Language and Computation · doi:10.1007/s11168-010-9068-8 · Cited by 24
Proceedings of the Ninth International Conference on Computational Semantics (IWCS 2011)
Bos, J., & Pulman, S. (2011) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 23
An inference-based model of word meaning in context as a paraphrase distribution
Moon, T., & Erk, K. (2013) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2483669.2483675 · Cited by 12
Self-citation
Cited by
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1023 · Cited by 1,364
Multimodal Distributional Semantics
Bruni, E., Tran, N. K., & Baroni, M. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4135 · Cited by 908
Distributional Models of Word Meaning
Lenci, A. (2018) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-030514-125254 · Cited by 447
From Word To Sense Embeddings: A Survey on Vector Representations of Meaning
Camacho-Collados, J., & Pilehvar, M. T. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11259 · Cited by 363
Metadata
- Authors
- Erk, K.
- Venue
- Language and Linguistics Compass
- Publisher
- Wiley
- Published
- 2012-10
- Volume
- 6
- Issue
- 10
- Pages
- 635-653
- ISSN
- 1749-818X (ISSN-L)
- OpenAlex
- W1597195725
- Cited by
- 321
- References
- 51
- FWCI
- 23.75
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.