The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
The Google Similarity Distance
Open accessAbstract
Words and phrases acquire meaning from the way they are used in society, from their relative semantics to other words and phrases. For computers, the equivalent of "society" is "database," and the equivalent of "use" is "a way to search the database". We present a new theory of similarity between words and phrases based on information distance and Kolmogorov complexity. To fix thoughts, we use the World Wide Web (WWW) as the database, and Google as the search engine. The method is also applicable to other search engines and databases. This theory is then applied to construct a method to automatically extract similarity, the Google similarity distance, of words and phrases from the WWW using Google page counts. The WWW is the largest database on earth, and the context information entered by millions of independent users averages out to provide automatic semantics of useful quality. We give applications in hierarchical clustering, classification, and language translation. We give examples to distinguish between colors and numbers, cluster names of paintings by 17th century Dutch masters and names of books by English novelists, the ability to understand emergencies and primes, and we demonstrate the ability to do a simple automatic English-Spanish translation. Finally, we use the WordNet database as an objective baseline against which to judge the performance of our method. We conduct a massive randomized trial in binary classification using support vector machines to learn categories based on our Google distance, resulting in an a mean agreement of 87 percent with the expert crafted WordNet categories
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
A Tutorial on Support Vector Machines for Pattern Recognition
Burges, C. J. C. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009715923555 · Cited by 16,429
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
An Introduction to Kolmogorov Complexity and Its Applications
Li, M., & Vitányi, P. (2019) · Texts in Computer Science · doi:10.1007/978-3-030-11298-1 · Cited by 3,681
A TUTORIAL ON SUPPORT VECTOR MACHINES FOR PATTERN RECOGNITION, DATA MINING AND KNOWLEDGE DISCOVERY
Burger, C. J. C. (1998) · Cited by 3,246
Three approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031
CYC: a large-scale investment in knowledge infrastructure
Lenat, D. B. (1995) · Communications of the ACM · doi:10.1145/219717.219745 · Cited by 1,946
Cited by
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02145-9 · Cited by 3,281
Sentiment Analysis and Opinion Mining
Zhang, L., & Liu, B. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_907 · Cited by 3,260
A Survey of Text Similarity Approaches
H.Gomaa, W., & A. Fahmy, A. (2013) · International Journal of Computer Applications · doi:10.5120/11638-7118 · Cited by 823
Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions
Shen, W., Wang, J., & Han, J. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2327028 · Cited by 761
Recent automatic text summarization techniques: a survey
Gambhir, M., & Gupta, V. (2017) · Artificial Intelligence Review · doi:10.1007/s10462-016-9475-9 · Cited by 727
Local and global algorithms for disambiguation to Wikipedia
Ratinov, L. (2024) · TIB Data Manager · doi:10.57702/yvubzlq1 · Cited by 596
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Measuring semantic similarity between words using web search engines
(2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242675 · Cited by 540
Sentiment Analysis: Mining Opinions, Sentiments, and Emotions
Liu, B. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108639286 · Cited by 362
Medelyan, O., Milne, D., Legg, C., & Witten, I. H. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.05.004 · Cited by 357
Using the Web to Obtain Frequencies for Unseen Bigrams
Keller, F., & Lapata, M. (2003) · Computational Linguistics · doi:10.1162/089120103322711604 · Cited by 350
Query expansion techniques for information retrieval: a survey
Azad, H. K., & Deepak, A. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.05.009 · Cited by 313
Evolution of Semantic Similarity—A Survey
Chandrasekaran, D., & Mago, V. (2022) · ACM Computing Surveys · doi:10.1145/3440755 · Cited by 303
Springer Handbook of Science and Technology Indicators
(2019) · Springer Handbooks · doi:10.1007/978-3-030-02511-3 · Cited by 300
Representation of the Online Tourism Domain in Search Engines
Xiang, Z., Wöber, K., & Fesenmaier, D. R. (2008) · Journal of Travel Research · doi:10.1177/0047287508321193 · Cited by 291
Opinion mining from online hotel reviews – A text summarization approach
Hu, Y.-H., Chen, Y.-L., & Chou, H.-L. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2016.12.002 · Cited by 277
PhishStorm: Detecting Phishing With Streaming Analytics
Marchal, S., Francois, J., State, R., & Engel, T. (2014) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2014.2377295 · Cited by 271
Aliguliyev, R. M. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.11.022 · Cited by 265
Paparo, G. D., & Martin-Delgado, M. A. (2012) · Scientific Reports · doi:10.1038/srep00444 · Cited by 136
Measuring neural representations with fMRI: practices and pitfalls
Davis, T., & Poldrack, R. A. (2013) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12156 · Cited by 132
C‐sanitized: A privacy model for document redaction and sanitization
Sánchez, D., & Batet, M. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23363 · Cited by 80
Automatic Construction of Subject Knowledge Graph based on Educational Big Data
Su, Y., & Zhang, Y. (2020) · Proceedings of the 2020 3rd International Conference on Big Data and Education · doi:10.1145/3396452.3396458 · Cited by 34
Improving accuracy for identifying related PubMed queries by an integrated approach
Lu, Z., & Wilbur, W. J. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.12.006 · Cited by 18
(2013)
Metadata
- Authors
- Cilibrasi, R. L., & Vitanyi, P. M. B.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2007-03
- Volume
- 19
- Issue
- 3
- Pages
- 370-383
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W2165897980
- Cited by
- 1,760
- References
- 24
- FWCI
- 125.68
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.