Siamese Recurrent Architectures for Learning Sentence Similarity
Open accessAbstract
We present a siamese adaptation of the Long Short-Term Memory (LSTM) network for labeled data comprised of pairs of variable-length sequences. Our model is applied to assess semantic similarity between sentences, where we exceed state of the art, outperforming carefully handcrafted features and recently proposed neural network systems of greater complexity. For these applications, we provide word-embedding vectors supplemented with synonymic information to the LSTMs, which use a fixed size vector to encode the underlying meaning expressed in a sentence (irrespective of the particular wording/syntax). By restricting subsequent operations to rely on a simple Manhattan metric, we compel the sentence representations learned by our model to form a highly structured space whose geometry reflects complex semantic relationships. Our results are the latest in a line of findings that showcase LSTMs as powerful language models capable of tasks requiring intricate understanding.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Siamese networks are widely applied in nonlinear metric learning across domains such as computer vision and speech recognition.
“Siamese networks are widely applied in nonlinear metric learning across domains such as computer vision and speech recognition.”
From Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches (2025) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 3× · see citing works
“Competitive methods for this data have all utilized both heterogeneous features (e.g. word overlap/similarity, negation modeling, sentence/phrase composition) as well as external resources (e.g. Wordnet (Miller 1995)), and a wide variety of learning algorithms have been applied (e.g. SVM, Random forest, k-Nearest Neig…”
Distributed Representations of Sentences and DocumentsPreprint
Le, Q. V., & Mikolov, T. (2014) · International Conference on Machine Learning · doi:10.48550/arxiv.1405.4053 · Cited by 10,307 · see citing works
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.01626 · Cited by 7,552 · see citing works
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · see citing works
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · see citing works
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036 · see citing works
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · see citing works
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120 · see citing works
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081 · see citing works
5.7 The WORDNET lexical database of English
Ooi, V. B. Y. (2019) · Edinburgh University Press eBooks · doi:10.1515/9781474471459-050 · Cited by 1,861 · see citing works
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · see citing works
Corpus-based and Knowledge-based Measures of Text Semantic Similarity
Mihalcea, R. F., Corley, C. D., & Strapparava, C. (2006) · University of North Texas Digital Library (University of North Texas) · Cited by 1,196 · see citing works
Deep Learning of Representations for Unsupervised and Transfer Learning.
Bengio, Y. (2011) · Cited by 885 · see citing works
On the Computational Power of Neural Nets
Siegelmann, H. T., & Sontag, E. D. (1995) · Journal of Computer and System Sciences · doi:10.1006/jcss.1995.1013 · Cited by 703 · see citing works
Variable Bandwidth and Local Linear Regression Smoothers
B, J. F. A., & Gijbels, I. (1992) · The Annals of Statistics · doi:10.1214/aos/1176348900 · Cited by 636 · see citing works
Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks
He, H., Gimpel, K., & Lin, J. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1181 · Cited by 378 · see citing works
Marelli, M., Bentivogli, L., Baroni, M., Raffaella Bernardi, B., Menini, S., & Zamparelli, R. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2001 · Cited by 378 · see citing works
*SEM 2013 shared task: Semantic Textual Similarity
Agirre, E., Cer, D. M., Diab, M., González-Agirre, A., & Guo, W. (2013) · Joint Conference on Lexical and Computational Semantics · Cited by 360 · see citing works
Learning Discriminative Projections for Text Similarity Measures
Yih, W.-T., Toutanova, K., Platt, J. C., & Meek, C. (2011) · Cited by 216 · see citing works
Illinois-LH: A Denotational and Distributional Approach to Semantics
Lai, A., & Hockenmaier, J. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2055 · Cited by 134 · see citing works
Zhao, J. B., Zhu, T., & Lan, M. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2044 · Cited by 114 · see citing works
Li, Y., Xu, L., Tian, F., et al. (2015) · Cited by 100 · see citing works
Bjerva, J., Bos, J., van der Goot, R., et al. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2114 · Cited by 92 · see citing works
Jiménez, S., Dueñas, G., Baquero, J., et al. (2014) · International Workshop on Semantic Evaluation (SemEval) · doi:10.3115/v1/s14-2131 · Cited by 68 · see citing works
Extracting Speaker-Specific Information with a Regularized Siamese Deep Network
Chen, K., & Salman, A. (2011) · Research Explorer (The University of Manchester) · Cited by 65 · see citing works
A Tableau Prover for Natural Logic and Language
Abzianidze, L. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1296 · Cited by 43 · see citing works
Semantic Parsing for Textual Entailment
Lien, E., & Kouylekov, M. (2015) · doi:10.18653/v1/w15-2205 · Cited by 24 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Socher, R. 2014. Recursive Deep Learning for Natural Language Processing and Computer Vision . Phd thesis, Stanford University.
Cited by
Sorted by citations
Learning Text Similarity with Siamese Recurrent Networks
Neculoiu, P., Versteegh, M., & Rotaru, M. (2016) · Proceedings of the 1st Workshop on Representation Learning for NLP · doi:10.18653/v1/w16-1617 · Cited by 341 · see citing works
“More recently, (Mueller and Thyagarajan, 2016) applied Siamese recurrent networks to learning semantic entailment.”
BackgroundDeep learning approaches for similarity computation: A survey
Yang, P., Wang, H., Yang, J., Qian, Z., Zhang, Y., & Lin, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3422484 · Cited by 28 · see citing works
Chen, Q., Du, J., Kim, S., Wilbur, W. J., & Lu, Z. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-1044-0 · Cited by 19 · see citing works
Siamese Networks for Weakly Supervised Human Activity Recognition
Sheng, T., & Huber, M. (2019) · 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) · doi:10.1109/smc.2019.8914045 · Cited by 21 · see citing works
Sheng, T., & Huber, M. (2025) · Sensors · doi:10.3390/s25134032 · Cited by 5 · see citing works
ODDA: An OODA-Driven Diverse Data Augmentation Framework for Low-Resource Relation Extraction
Zhong, Y., Gao, Y., Zhang, X., & Wang, H. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.15 · Cited by 3 · see citing works
“(1) WordNet Synonym Substitution (WSS) (Mueller and Thyagarajan, 2016): re-placing tokens with synonyms from WordNet; (2) Word Embedding Substitution (WES) (Jiao et al., 2020): replacing tokens with contextual embed-dings from BERT; (3) LAMBADA (Anaby-Tavor et al., 2020): fine-tuning generative models to generate cand…”
Influential
- Authors
- Mueller, J., & Thyagarajan, A.
- Affiliations
- Massachusetts Institute of Technology (US)
- Ramaiah Institute of Technology (IN)
- ORCID
- Mueller, Jonas
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2016-03-05
- Volume
- 30
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2508865106
- Cited by
- 935
- References
- 33
- Citations by year
- 2026: 15 · 2025: 38 · 2024: 56 · 2023: 90 · 2022: 93 · 2021: 170 · 2020: 162 · 2019: 150 · 2018: 107 · 2017: 47 · 2016: 5 · 2015: 1 · 2012: 1
- FWCI
- 54.46
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- sentence similarity, Siamese network, long short-term memory, semantic similarity, word embeddings, Manhattan metric, sentence representation learning, paraphrase detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
