The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Evolution of Semantic Similarity—A Survey
Open accessAbstract
Estimating the semantic similarity between text data is one of the challenging and open research problems in the field of Natural Language Processing (NLP). The versatility of natural language makes it difficult to define rule-based methods for determining semantic similarity measures. To address this issue, various semantic similarity methods have been proposed over the years. This survey article traces the evolution of such methods beginning from traditional NLP techniques such as kernel-based methods to the most recent research work on transformer-based models, categorizing them based on their underlying principles as knowledge-based, corpus-based, deep neural network–based methods, and hybrid methods. Discussing the strengths and weaknesses of each method, this survey provides a comprehensive view of existing systems in place for new researchers to experiment and develop innovative ideas to address the issue of semantic similarity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 4×
“The Lesk measure[10], assigns a value of relatedness between two words based on the overlap of words in their gloss and the glosses of the concepts they are related to in an ontology like WordNet[43].”
MethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218
Verbs semantics and lexical selection
Wu, Z., & Palmer, M. (1994) · Proceedings of the 32nd annual meeting on Association for Computational Linguistics - · doi:10.3115/981732.981751 · Cited by 3,136
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,656
DBpedia - A crystallization point for the Web of Data
Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., & Hellmann, S. (2009) · Journal of Web Semantics · doi:10.1016/j.websem.2009.07.002 · Cited by 2,137
The University of South Florida free association, rhyme, and word fragment norms
Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195588 · Cited by 1,929
Development and application of a metric on semantic nets
Rada, R., Mili, H., Bicknell, E., & Blettner, M. (1989) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/21.24528 · Cited by 1,824
Producing high-dimensional semantic spaces from lexical co-occurrence
Lund, K., & Burgess, C. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204766 · Cited by 1,811
The Google Similarity Distance
Cilibrasi, R. L., & Vitanyi, P. M. B. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.48 · Cited by 1,760
Contextual correlates of semantic similarity
Miller, G. A., & Charles, W. G. (1991) · Language and Cognitive Processes · doi:10.1080/01690969108406936 · Cited by 1,594
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation
Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., & Specia, L. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2001 · Cited by 1,562
Contextual correlates of synonymy
Rubenstein, H., & Goodenough, J. B. (1965) · Communications of the ACM · doi:10.1145/365628.365657 · Cited by 1,425
Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1023 · Cited by 1,364
A Decomposable Attention Model for Natural Language Inference
Parikh, A., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,296
YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia
Hoffart, J., Suchanek, F. M., Berberich, K., & Weikum, G. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2012.06.001 · Cited by 1,232
The WaCky wide web: a collection of very large linguistically processed web-crawled corpora
Baroni, M., Bernardini, S., Ferraresi, A., & Zanchetta, E. (2009) · Language Resources and Evaluation · doi:10.1007/s10579-009-9081-4 · Cited by 1,200
SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation
Hill, F., Reichart, R., & Korhonen, A. (2015) · Computational Linguistics · doi:10.1162/coli_a_00237 · Cited by 1,175
An approach for measuring semantic similarity between words using multiple information sources
Li, Y., Bandar, Z. A., & McLean, D. (2003) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2003.1209005 · Cited by 1,090
Dependency-Based Word Embeddings
Levy, O., & Goldberg, Y. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.3115/v1/p14-2050 · Cited by 1,047
Determining semantic similarity among entity classes from different ontologies
Rodriguez, M. A., & Egenhofer, M. J. (2003) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2003.1185844 · Cited by 890
A study on similarity and relatedness using distributional and WordNet-based approaches
Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., Paşca, M., & Soroa, A. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics on - NAACL '09 · doi:10.3115/1620754.1620758 · Cited by 860
Placing search in context: the concept revisited
Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., & Ruppin, E. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372094 · Cited by 839
WikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829
Sentence similarity based on semantic nets and corpus statistics
Li, Y., McLean, D., Bandar, Z. A., O'Shea, J. D., & Crockett, K. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.130 · Cited by 819
ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
Sun, Y., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., & Wang, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6428 · Cited by 766
Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features
Pagliardini, M., Gupta, P., & Jaggi, M. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1049 · Cited by 678
Question Answering with Subgraph Embeddings
Bordes, A., Chopra, S., & Weston, J. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1067 · Cited by 635
Measures of semantic similarity and relatedness in the biomedical domain
Pedersen, T., Pakhomov, S. V. S., Patwardhan, S., & Chute, C. G. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2006.06.004 · Cited by 566
Evaluation methods for unsupervised word embeddings
Schnabel, T., Labutov, I., Mimno, D., & Joachims, T. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1036 · Cited by 540
SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation
Agirre, E., Banea, C., Cer, D., Diab, M., Gonzalez-Agirre, A., Mihalcea, R., Rigau, G., & Wiebe, J. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1081 · Cited by 519
context2vec: Learning Generic Context Embedding with Bidirectional LSTM
Melamud, O., Goldberger, J., & Dagan, I. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1006 · Cited by 513
SemEval-2014 Task 10: Multilingual Semantic Textual Similarity
Agirre, E., Banea, C., Cardie, C., Cer, D., Diab, M., Gonzalez-Agirre, A., Guo, W., Mihalcea, R., Rigau, G., & Wiebe, J. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2010 · Cited by 490
SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability
Agirre, E., Banea, C., Cardie, C., Cer, D., Diab, M., Gonzalez-Agirre, A., Guo, W., Lopez-Gazpio, I., Maritxalar, M., Mihalcea, R., et al. (2015) · Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015) · doi:10.18653/v1/s15-2045 · Cited by 466
Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees
Moschitti, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11871842_32 · Cited by 452
Ontology-based semantic similarity: A new feature-based approach
Sánchez, D., Batet, M., Isern, D., & Valls, A. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.01.082 · Cited by 383
From Word To Sense Embeddings: A Survey on Vector Representations of Meaning
Camacho-Collados, J., & Pilehvar, M. T. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11259 · Cited by 363
Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement
He, H., & Lin, J. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1108 · Cited by 232
Computing Semantic Similarity of Concepts in Knowledge Graphs
Zhu, G., & Iglesias, C. A. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2610428 · Cited by 232
SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity
Gerz, D., Vulić, I., Hill, F., Reichart, R., & Korhonen, A. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1235 · Cited by 218
Learning Grounded Meaning Representations with Autoencoders
Silberer, C., & Lapata, M. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1068 · Cited by 217
Semantic text classification: A survey of past and recent advances
Altınel, B., & Ganiz, M. C. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.08.001 · Cited by 192
Camacho-Collados, J., Pilehvar, M. T., & Navigli, R. (2016) · Artificial Intelligence · doi:10.1016/j.artint.2016.07.005 · Cited by 188
Tree Kernels for Semantic Role Labeling
Moschitti, A., Pighin, D., & Basili, R. (2008) · Computational Linguistics · doi:10.1162/coli.2008.34.2.193 · Cited by 157
Knowledge-enhanced document embeddings for text classification
Sinoara, R. A., Camacho-Collados, J., Rossi, R. G., Roberto Navigli, A., & Rezende, S. O. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.10.026 · Cited by 128
Gao, J., Zhang, B.-W., & Chen, X.-H. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.11.009 · Cited by 116
Mohamed, M., & Oussalah, M. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.04.003 · Cited by 116
DLS$@$CU: Sentence Similarity from Word Alignment and Semantic Vector Composition
Sultan, M. A., Bethard, S. J., & Sumner, T. (2015) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s15-2027 · Cited by 115
Dict2vec : Learning Word Embeddings using Lexical Dictionaries
Tissier, J., Gravier, C., & Habrard, A. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1024 · Cited by 107
NASARI: a Novel Approach to a Semantically-Aware Representation of Items
Camacho-Collados, J., Pilehvar, M. T., & Roberto Navigli, A. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1059 · Cited by 106
From senses to texts: An all-in-one graph-based approach for measuring semantic similarity
Pilehvar, M. T., & Roberto Navigli, A. (2015) · Artificial Intelligence · doi:10.1016/j.artint.2015.07.005 · Cited by 98
Lastra-Díaz, J. J., Goikoetxea, J., Taieb, M. A. H., García-Serrano, A. M., Aouicha, M. B., & Agirre, E. (2019) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2019.07.010 · Cited by 95
Wikipedia-based information content and semantic similarity computation
Jiang, Y., Bai, W., Zhang, X., et al. (2016) · Information Processing & Management · doi:10.1016/j.ipm.2016.09.001 · Cited by 82
Tian, J., Zhou, Z., Lan, M., & Wu, Y. (2017) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s17-2028 · Cited by 81
Tien, N. H., Nguyen, L.-M., Yamasaki, T., & Izuha, T. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.102090 · Cited by 80
An Efficient Framework for Sentence Similarity Modeling
Quan, Z., Wang, Z.-J., Le, Y., Yao, B., Li, K., & Yin, J. P. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2899494 · Cited by 73
Shao, Y. (2017) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s17-2016 · Cited by 71
Computing inter-document similarity with Context Semantic Analysis
Benedetti, F., Beneventano, D., Bergamaschi, S., & Simonini, G. (2018) · Information Systems · doi:10.1016/j.is.2018.02.009 · Cited by 70
DLS$@$CU: Sentence Similarity from Word Alignment
Sultan, M. A., Bethard, S. J., & Sumner, T. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2039 · Cited by 68
Syntactic, Semantic and Sentiment Analysis: The Joint Effect on Automated Essay Evaluation
Janda, H. K., Pawar, A., Du, S., & Mago, V. (2019) · IEEE Access · doi:10.1109/access.2019.2933354 · Cited by 67
Self-citationKim, S., Fiorini, N., Wilbur, W. J., & Lu, Z. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.09.014 · Cited by 64
Word n-gram attention models for sentence similarity and inference
López-Gazpio, I., Maritxalar, M., Lapata, M., & Agirre, E. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.04.054 · Cited by 62
Lastra-Díaz, J. J., García-Serrano, A. M., Batet, M., Fernández, M., & Chirigati, F. (2017) · Information Systems · doi:10.1016/j.is.2017.02.002 · Cited by 57
A survey of semantic relatedness evaluation datasets and procedures
Taieb, M. A. H., Zesch, T., & Aouicha, M. B. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09796-3 · Cited by 51
Multi-sense embeddings through a word sense disambiguation process
Ruas, T., Grosky, W. I., & Aizawa, A. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.06.026 · Cited by 50
A novel sentence similarity measure for semantic-based expert systems
Lee, M.-C. (2010) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.10.043 · Cited by 49
A resource-light method for cross-lingual semantic textual similarity
Glavaš, G., Franco-Salvador, M., Ponzetto, S. P., & Rosso, P. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.11.041 · Cited by 48
Computing semantic similarity based on novel models of semantic representation using Wikipedia
Qu, R., Fang, Y., Bai, W., & Jiang, Y. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.07.002 · Cited by 43
Challenging the Boundaries of Unsupervised Learning for Semantic Similarity
Pawar, A., & Mago, V. (2019) · IEEE Access · doi:10.1109/access.2019.2891692 · Cited by 40
Self-citationUESTS: An Unsupervised Ensemble Semantic Textual Similarity Method
Hassan, B., Abdelrahman, S., Bahgat, R., & Farag, I. (2019) · IEEE Access · doi:10.1109/access.2019.2925006 · Cited by 32
ACV-tree: A New Method for Sentence Similarity Modeling
Le, Y., Wang, Z.-J., Quan, Z., He, J., & Yao, B. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/575 · Cited by 30
Deep Learning in Semantic Kernel Spaces
Croce, D., Filice, S., Castellucci, G., & Basili, R. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1032 · Cited by 20
Sentence similarity based on semantic kernels for intelligent text retrieval
Amir, S., Tanasescu, A., & Zighed, A. D. (2016) · Journal of Intelligent Information Systems · doi:10.1007/s10844-016-0434-3 · Cited by 19
Cited by
Distributional Semantics: Meaning Through Culture and Interaction
Contreras Kallens, P., & Christiansen, M. H. (2025) · Topics in Cognitive Science · doi:10.1111/tops.12771 · Cited by 17
The Geometry of Concepts: Sparse Autoencoder Feature Structure
Li, Y., Michaud, E. J., Baek, D. D., Engels, J., Sun, X., & Tegmark, M. (2024) · arXiv · doi:10.48550/arxiv.2410.19750 · Cited by 14
LEGO-GraphRAG: Modularizing Graph-Based Retrieval-Augmented Generation for Design Space Exploration
Cao, Y., Gao, Z., Li, Z., Xie, X., Zhou, S. K., & Xu, J. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3748191.3748194 · Cited by 11
Gilani, K., Verket, M., Peters, C., Dumontier, M., Rocca, H.-P. B.-L., & Urovi, V. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110745 · Cited by 5
Metadata
- Authors
- Chandrasekaran, D., & Mago, V.
- Venue
- ACM Computing Surveys
- Published
- 2021-02-18
- Volume
- 54
- Issue
- 2
- Pages
- 1-37
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3021792233
- Cited by
- 303
- References
- 82
- FWCI
- 15.79
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.