Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
We consider the problem of learning general-purpose, paraphrastic sentence embeddings in the setting of Wieting et al. (2016b).We use neural machine translation to generate sentential paraphrases via back-translation of bilingual sentence pairs.We evaluate the paraphrase pairs by their ability to serve as training data for learning paraphrastic sentence embeddings.We find that the data quality is stronger than prior work based on bitext and on par with manually-written English paraphrase pairs, with the advantage that our approach can scale up to generate large training sets for many languages and domains.We experiment with several language pairs and data sources, and develop a variety of data filtering techniques.In the process, we explore how neural machine translation output differs from humanwritten sentences, finding clear differences in length, the amount of repetition, and the use of rare words. 1 1 Generated paraphrases and code are available at http: //ttic.uchicago.edu/˜wieting.R: We understand that has already commenced, but there is a long way to go.T: This situation has already commenced, but much still needs to be done.R: The restaurant is closed on Sundays.No breakfast is available on Sunday mornings
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · cited 1×
“Other work in learning general purpose sentence embeddings has used autoencoders (Socher et al., 2011; Hill et al., 2016), encoder-decoder architectures (Kiros et al., 2015), or other learning frameworks (Le and Mikolov, 2014; Pham et al., 2015).”
BackgroundOn Using Very Large Target Vocabulary for Neural Machine Translation
Jean, S., Cho, K., Memisevic, R., & Bengio, Y. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1001 · Cited by 862
Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources
Dolan, B., Quirk, C., & Brockett, C. (2004) · Proceedings of the 20th international conference on Computational Linguistics - COLING '04 · doi:10.3115/1220355.1220406 · Cited by 728 · cited 1×
“There is a rich history of research in generating or finding naturally-occurring sentential paraphrases (Barzi-lay and McKeown, 2001; Dolan et al., 2004; Dolan and Brockett, 2005; Quirk et al., 2004; Zhao et al., 2010; Coster and Kauchak, 2011; Xu et al., 2014, 2015).”
BackgroundLearning Distributed Representations of Sentences from Unlabelled Data
Hill, F., Cho, K., & Korhonen, A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1162 · Cited by 566 · cited 3×
“Wieting et al. (2016b) and Hill et al. (2016) provide many empirical comparisons to this prior work.”
BackgroundResult
Cited by
From Recognition to Cognition: Visual Commonsense Reasoning
Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00688 · Cited by 769
“Additionally, we found that even after training on these corpora, the model would struggle with paraphrases, so we also translated SNLI sentences from English to German and back using the Nematus machine translation system [81, 73].”
MethodAdversarial Example Generation with Syntactically Controlled Paraphrase Networks
Iyyer, M., Wieting, J., Gimpel, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1170 · Cited by 615
Self-citationAn efficient framework for learning sentence representations
Logeswaran, L., & Lee, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02893 · Cited by 427
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Yu, A. W., Dohan, D., Luong, M.-T., Zhao, R., Chen, K., Norouzi, M., & Le, Q. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09541 · Cited by 368
Style transfer through back-translation
Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., & Black, A. W. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1080 · Cited by 348
“More recently, Mallinson et al. (2017) and Wieting et al. (2017) showed how neural back-translation can be used to generate paraphrases.”
Method
Metadata
- Authors
- Wieting, J., Mallinson, J., & Gimpel, K.
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Published
- 2017
- Pages
- 274-285
- OpenAlex
- W2622000134
- Cited by
- 19
- References
- 5
- FWCI
- 1.65
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext. https://app.cochat.ai/papers/10.18653/v1/d17-1026
- The paper
- Wieting, J., Mallinson, J., & Gimpel, K. (2017). Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/d17-1026
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.