The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Similarity-Based Models of Word Cooccurrence Probabilities
Open accessAbstract
In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations “eat a peach” and ”eat a beach” is more likely. Statistical NLP methods determine the likelihood of a word combination from its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in any given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on “most similar” words.We describe probabilistic word association models based on distributional word similarity, and apply them to two tasks, language modeling and pseudo-word disambiguation. In the language modeling task, a similarity-based model is used to improve probability estimates for unseen bigrams in a back-off language model. The similarity-based method yields a 20% perplexity improvement in the prediction of unseen bigrams and statistically significant reductions in speech-recognition error.We also compare four similarity-based estimation methods against back-off and maximum-likelihood estimation methods on a pseudo-word sense disambiguation task in which we controlled for both unigram and bigram frequency to avoid giving too much weight to easy-to-disambiguate high-frequency configurations. The similarity-based methods perform up to 40% better on this particular task.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Information theory and statistics
(1959) · Journal of the Franklin Institute · doi:10.1016/0016-0032(59)90368-0 · Cited by 7,094
Divergence measures based on the Shannon entropy
Lin, J. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.61115 · Cited by 5,140
Introduction to WordNet: An On-line Lexical Database*
Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990) · International Journal of Lexicography · doi:10.1093/ijl/3.4.235 · Cited by 4,197
An Information-Theoretic Definition of Similarity
Lin, D. (1998) · International Conference on Machine Learning · Cited by 3,790
THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS
GOOD, I. J. (1953) · Biometrika · doi:10.1093/biomet/40.3-4.237 · Cited by 3,437
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274
(1992) · doi:10.5555/176313.176316 · Cited by 2,892
Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy
Jiang, J. J., & Conrath, D. W. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9709008 · Cited by 2,743
Instance-based learning algorithms
Aha, D. W., Kibler, D., & Albert, M. K. (1991) · Machine Learning · doi:10.1007/bf00153759 · Cited by 2,163
Atkeson, C. G., Moore, A. W., & Schaal, S. (1997) · Artificial Intelligence Review · doi:10.1023/a:1006559212014 · Cited by 1,671
Estimation of probabilities from sparse data for the language model component of a speech recognizer
Katz, S. (1987) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1987.1165125 · Cited by 1,637
Information Theory and Statistics.
Dwass, M., & Kullback, S. (1960) · The American Mathematical Monthly · doi:10.2307/2309734 · Cited by 1,537
Elements of Information Theory
Phoenix, S. J. D. (1992) · Journal of Modern Optics · doi:10.1080/09500349214551641 · Cited by 1,380
Stanfill, C., & Waltz, D. (1986) · Communications of the ACM · doi:10.1145/7902.7906 · Cited by 1,218
Pattern Classification and Scene Analysis.
Grenander, U., Duda, R. O., & Hart, P. E. (1974) · Journal of the American Statistical Association · doi:10.2307/2286028 · Cited by 1,006
Distributional clustering of English words
Pereira, F., Tishby, N., & Lee, L. (1993) · Proceedings of the 31st annual meeting on Association for Computational Linguistics - · doi:10.3115/981574.981598 · Cited by 974
Self-citationA stochastic parts program and noun phrase parser for unrestricted text
Church, K. W. (1988) · Proceedings of the second conference on Applied natural language processing - · doi:10.3115/974235.974260 · Cited by 957
Interpolated estimation of Markov source parameters from sparse data
Jelinek, F. (1980) · Medical Entomology and Zoology · Cited by 823
An empirical study of smoothing techniques for language modeling
Chen, S. F., & Goodman, J. (1996) · Proceedings of the 34th annual meeting on Association for Computational Linguistics - · doi:10.3115/981863.981904 · Cited by 770
Explorations in Automatic Thesaurus Discovery
Grefenstette, G. (1994) · Springer eBooks · doi:10.1007/978-1-4615-2710-7 · Cited by 749
Witten, I. H., & Bell, T. C. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.87000 · Cited by 729
Word-sense disambiguation using statistical models of Roget's categories trained on large corpora
Yarowsky, D. (1992) · Proceedings of the 14th conference on Computational linguistics - · doi:10.3115/992133.992140 · Cited by 663
An Empirical Study of Smoothing Techniques for Language Modeling
Chen, S. F., & Goodman, J. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9606011 · Cited by 612
Noun classification from predicate-argument structures
Hindle, D. (1990) · Proceedings of the 28th annual meeting on Association for Computational Linguistics - · doi:10.3115/981823.981857 · Cited by 545
Advances in Speech Signal Processing
Furui, S., & Sondhi, M. M. (1991) · Cited by 435
Integrating multiple knowledge sources to disambiguate word sense
Ng, H. T., & Lee, H. B. (1996) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/981863.981869 · Cited by 408
Atkeson, C. G., Moore, A. W., & Schaal, S. (1997) · Lazy Learning · doi:10.1007/978-94-017-2053-3_2 · Cited by 308
Schütze, H. (1992) · Cited by 300
Diversity: its measurement, decomposition, apportionment and analysis
Rao, C. R. (1982) · Cited by 258
Using syntactic dependency as local context to resolve word sense ambiguity
Lin, D. (1997) · Proceedings of the 35th annual meeting on Association for Computational Linguistics - · doi:10.3115/976909.979626 · Cited by 242
Church, K. W., & Gale, W. A. (1991) · Computer Speech & Language · doi:10.1016/0885-2308(91)90016-j · Cited by 234
Disambiguating Noun Groupings with Respect to WordNet Senses
Resnik, P. (1999) · Text, Speech and Language Technology · doi:10.1007/978-94-017-2390-9_6 · Cited by 210
Schütze, H. (1992) · neural information processing systems · Cited by 205
Improved clustering techniques for class-based statistical language modelling
Kneser, R., & Ney, H. (1993) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1993-229 · Cited by 193
Aggregate and mixed-order Markov models for statistical language processingPreprint
Saul, L. K., & Pereira, F. M. B. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9706007 · Cited by 160
Self-citationSimilarity-based methods for word sense disambiguation
Dagan, I., Lee, L., & Pereira, F. M. B. (1997) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/976909.979625 · Cited by 158
Self-citationSchütze, H. (2003) · ACM/IEEE Conference on Supercomputing (SC) · doi:10.1109/superc.1992.236684 · Cited by 155
Contextual word similarity and estimation from sparse data
Dagan, I., Marcus, S., & Markovitch, S. (1993) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981574.981596 · Cited by 124
Self-citationSimilarity-based estimation of word cooccurrence probabilities
Dagan, I., Pereira, F. M. B., & Lee, L. (1994) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981732.981770 · Cited by 122
Self-citationWork on Statistical Methods for Word Sense Disambiguation
Gale, W. A., Church, K., & Yarowsky, D. (1992) · Cited by 89
Exemplar-Based Word Sense Disambiguation: Some Recent ImprovementsPreprint
Ng, H. T. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9706010 · Cited by 80
Similarity-Based Approaches to Natural Language ProcessingPreprint
Lee, L. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9708011 · Cited by 75
Self-citationWordNet and Distributional Analysis: A Class-based Approach to Lexical Discovery
Resnik, P. (1992) · Cited by 71
Integrating Multiple Knowledge Sources to Disambiguate Word Sense: An Exemplar-Based ApproachPreprint
Ng, H. T., & Lee, H. B. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9606032 · Cited by 70
Statistical sense disambiguation with relatively small corpora using dictionary definitions
Luk, A. K. (1995) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981658.981683 · Cited by 61
Finch, S. (1995) · ERA · Cited by 60
Hierarchical clustering of words
Ushioda, A. (1996) · International Conference on Computational Linguistics · doi:10.3115/993268.993390 · Cited by 53
Memory-based learning: using similarity for smoothing
Zavrel, J., & Daelemans, W. M. P. (1997) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/976909.979673 · Cited by 43
Memory-Based Learning: Using Similarity for SmoothingPreprint
Zavrel, J., & Daelemans, W. M. P. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9705010 · Cited by 31
Similarity-based methods for word sense disambiguation
Dagan, I., Lee, L., & Pereira, F. M. B. (1997) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/979617.979625 · Cited by 28
Self-citation
Cited by
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
Evaluating WordNet-based Measures of Lexical Semantic Relatedness
Budanitsky, A., & Hirst, G. (2006) · Computational Linguistics · doi:10.1162/089120106776173093 · Cited by 1,448
KEYWORD EXTRACTION FROM A SINGLE DOCUMENT USING WORD CO-OCCURRENCE STATISTICAL INFORMATION
MATSUO, Y., & ISHIZUKA, M. (2004) · International Journal on Artificial Intelligence Tools · doi:10.1142/s0218213004001466 · Cited by 788
Measures of distributional similarity
Lee, L. (1999) · Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics - · doi:10.3115/1034678.1034693 · Cited by 583
Self-citationA word at a time: computing word relatedness using temporal semantic analysis
Radinsky, K., Agichtein, E., Gabrilovich, E., & Markovitch, S. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963455 · Cited by 405
Metadata
- Authors
- Dagan, I., Lee, L., & Pereira, F. C. N.
- Venue
- Machine Learning
- Published
- 1999-02
- Volume
- 34
- Issue
- 1-3
- Pages
- 43-69
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W1593239840
- Cited by
- 375
- References
- 53
- FWCI
- 15.06
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.