Analysis of sentence embedding models using prediction tasks in natural language processing

Long Short-Term Memory

Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works

Efficient Estimation of Word Representations in Vector Space

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works

Distributed Representations of Words and Phrases and their Compositionality

Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works

BLEU: a method for automatic evaluation of machine translation

Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works

Sequence to Sequence Learning with Neural Networks

Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works

Neural Machine Translation by Jointly Learning to Align and Translate

Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works

Untitled work

(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works

Speech recognition with deep recurrent neural networks

Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · see citing works

Untitled work

(2011) · Cited by 6,993 · see citing works

Software Framework for Topic Modelling with Large Corpora

Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,766 · see citing works

NLTK

Loper, E., & Bird, S. (2002) · doi:10.3115/1118108.1118117 · Cited by 3,248 · see citing works

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081 · see citing works

NLTK: The Natural Language ToolkitPreprint

Loper, E., & Bird, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205028 · Cited by 1,893 · see citing works

Neural Word Embedding as Implicit Matrix Factorization

Levy, O., & Goldberg, Y. (2014) · neural information processing systems · Cited by 1,581 · see citing works

Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors

Baroni, M., Dinu, G., & Kruszewski, G. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1023 · Cited by 1,391 · see citing works

Distributed representations, simple recurrent networks, and grammatical structure

Elman, J. L. (1991) · Machine Learning · doi:10.1007/bf00114844 · Cited by 1,354 · see citing works

Skip-Thought Vectors

Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · see citing works

Torch7: A Matlab-like Environment for Machine Learning

Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,241 · see citing works

Semi-supervised Sequence Learning

Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994 · see citing works

Visualizing and Understanding Recurrent Networks

Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845 · see citing works

Learning Distributed Representations of Sentences from Unlabelled Data

Hill, F., Cho, K., & Korhonen, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1162 · Cited by 566 · see citing works

A Hierarchical Neural Autoencoder for Paragraphs and Documents

Li, J., Luong, M.-T., & Jurafsky, D. (2015) · doi:10.3115/v1/p15-1107 · Cited by 507 · see citing works

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint

Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., & Goldberg, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04207 · Cited by 374 · see citing works

Does String-Based Neural MT Learn Source Syntax?

Shi, X., Padhi, I., & Knight, K. K. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1159 · Cited by 302 · see citing works

NLTK: The Natural Language Toolkit

Bird, S., & Loper, E. (2002) · Minerva Access (University of Melbourne) · Cited by 244 · see citing works

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint

Adi, Y. (2024) · arXiv (Cornell University) · doi:10.57702/tcr06vve · Cited by 228 · see citing works

Do Supervised Distributional Methods Really Learn Lexical Inference Relations?

Levy, O., Remus, S., Biemann, C., & Dagan, I. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1098 · Cited by 211 · see citing works

Representation of Linguistic Form and Function in Recurrent Neural Networks

Kádár, Á., Chrupała, G., & Alishahi, A. (2017) · Computational Linguistics · doi:10.1162/coli_a_00300 · Cited by 142 · see citing works

Probing for semantic evidence of composition by means of simple classification tasks

Ettinger, A., Elgohary, A., & Resnik, P. (2016) · Workshop on Evaluating Vector-Space Representations for NLP · doi:10.18653/v1/w16-2524 · Cited by 131 · see citing works

Investigating gated recurrent networks for speech synthesis

Wu, Z., & King, S. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472657 · Cited by 123 · see citing works

Distributed Representations, Simple Recurrent Networks, And Grammatical Structure

Elman, J. L. (1991) · Machine Learning · doi:10.1023/a:1022699029236 · Cited by 84 · see citing works

Why Neural Translations are the Right Length

Shi, X., Knight, K. K., & Yüret, D. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1248 · Cited by 67 · see citing works

A practical and linguistically-motivated approach to compositional distributional semantics

Paperno, D., Pham, N. T., & Baroni, M. (2014) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/v1/p14-1009 · Cited by 63 · see citing works

Visual Analysis of Hidden State Dynamics in Recurrent Neural NetworksPreprint

Strobelt, H., Gehrmann, S., Huber, B., Pfister, H., & Rush, A. M. (2016) · RePEc: Research Papers in Economics · Cited by 43 · see citing works

Software Framework for Topic Modelling with Large Corpora

Radim, Ř., & Petr, S. (2010) · Veřejné služby Informačního systému (Masarykiana Brunensis Universitas) · Cited by 24 · see citing works

rnn : Recurrent Library for TorchPreprint

Léonard, N., Waghmare, S., Wang, Y., & Kim, J.-H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07889 · Cited by 24 · see citing works

Analysis Methods in Neural Language Processing: A Survey

Belinkov, Y., & Glass, J. R. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00254 · Cited by 485 · see citing works

KnowIME: A System to Construct a Knowledge Graph for Intelligent Manufacturing Equipment

Yan, H., Yang, J., & Wan, J. (2020) · IEEE Access · doi:10.1109/access.2020.2977136 · Cited by 78 · see citing works

Exploring Semantic Properties of Sentence Embeddings

Zhu, X., Li, T., & de Melo, G. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2100 · Cited by 65 · see citing works

Predictive Student Modeling in Game-Based Learning Environments with Word Embedding Representations of Reflection

Geden, M., Emerson, A., Carpenter, D., Rowe, J., Azevedo, R., & Lester, J. C. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-020-00220-4 · Cited by 46 · see citing works

Transformer models for enhancing AttnGAN based text to image generation

Naveen, S., Kiran, M. S. S. R., Indupriya, M., Manikanta, T., & Sudeep, P. V. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104284 · Cited by 40 · see citing works

To Reverse the Gradient or Not: an Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition

Adi, Y., Zeghidour, N., Collobert, R., Usunier, N., Liptchinsky, V., & Synnaeve, G. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682468 · Cited by 35 · see citing works

textless-lib: a Library for Textless Spoken Language Processing

Kharitonov, E., Copet, J., Lakhotia, K., Nguyen, T. A., Tomasello, P., Lee, A., Elkahky, A. M., Hsu, W.-N., Mohamed, A., Emmanuel Dupoux, A., et al. (2022) · doi:10.18653/v1/2022.naacl-demo.1 · Cited by 28 · see citing works

Acoustic and temporal representations in convolutional neural network models of prosodic events

Stehwien, S., Schweitzer, A., & Vu, N. T. (2020) · Speech Communication · doi:10.1016/j.specom.2020.10.005 · Cited by 12 · see citing works

Naturalistic Causal Probing for Morpho-Syntax

Amini, A., Pimentel, T., Meister, C., & Cotterell, R. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00554 · Cited by 8 · see citing works

Zero-Shot Multi-Label Topic Inference with Sentence Encoders and LLMs

Sarkar, S., Feng, D., & Santu, S. K. K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1008 · Cited by 5 · see citing works

Opening the NLP Blackbox - Analysis and Evaluation of NLP Models

Mannarswamy, S. S., & Chidambaram, S. (2020) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3430984.3431969 · Cited by 3 · see citing works

Text Data Labelling using Transformer based Sentence Embeddings and Text Similarity for Text Classification

Chakrabarty, A. A. (2022) · International Journal on Natural Language Computing · doi:10.5121/ijnlc.2022.11201 · Cited by 3 · see citing works

Defense of the Ancients (DOTA 2)-Draft Recommendation System

Mohammed, Y., Srinivasan, S., Iyer, S., & Nagarajan, A. (2022) · International Conference on Trends in Electronics and Informatics (ICOEI) · doi:10.1109/icoei53556.2022.9776822 · Cited by 2 · see citing works

Constrained Consistency Modeling for Attributed Network Embedding

Zang, X., Yang, B. X., Yang, S., & Chen, H. (2019) · IEEE Access · doi:10.1109/access.2019.2959816 · Cited by 1 · see citing works

Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities Without Label Information

Joung, Y., Lee, S., & Choi, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-97-8702-9_21 · Cited by 1 · see citing works

Subword-based Sentence Representation Model for Sentiment Classification

Cho, D., Lee, H., & Kang, S.-S. (2020) · International Conference on Smart Media and Applications (SMA) · doi:10.1145/3426020.3426046 · Cited by 1 · see citing works

An Applied Computational Linguistics Approach to Clustering Scientific Research Papers

Vora, A., Mishra, M., & Muthukumar, S. (2023) · Advances in science and technology · doi:10.4028/p-s8m133

Topic aware probing: From sentence length prediction to idiom identification how reliant are neural language models on topic?

Nedumpozhimana, V., & Kelleher, J. D. (2024) · Natural language processing. · doi:10.1017/nlp.2024.43

How Can We Make Language Models Better at Handling the Diversity and Variability of Natural Languages ?

Müller, B. (2022) · theses.fr (ABES) · doi:10.70675/6984dc63z4731z4eabzb3fazaf5d012fe143

Word Meaning Representation in Neural Language Models : Lexical Polysemy and Semantic Relationships

Soler, A. G. (2021) · theses.fr (ABES) · doi:10.70675/99a10411zc05ez4135z8f31z13573e0fd4d1

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction TasksPreprint

Adi, Y. (2024) · arXiv (Cornell University) · doi:10.57702/tcr06vve · Cited by 228 · see citing works

What you can cram into a single vector: Probing sentence embeddings for linguistic propertiesPreprint

Conneau, A., Kruszewski, G., Lample, G., Barrault, L., & Baroni, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01070 · Cited by 315 · see citing works

Word and Relation Embedding for Sentence Representation

(2017) · Arizona State University Library Digital Repository (Arizona State University)

A Word Embeddings Model for Sentence Similarity

Mijangos, V., Sierra, G., & Herrera, A. (2016) · Research in Computing Science · doi:10.13053/rcs-117-1-5 · Cited by 4 · see citing works

Empirical Linguistic Study of Sentence Embeddings

Krasnowska-Kieraś, K., & Wróblewska, A. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1573 · Cited by 44 · see citing works