The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Open accessAbstract
We introduce an architecture to learn joint multilingual sentence representations for 93 languages, belonging to more than 30 different families and written in 28 different scripts. Our system uses a single BiLSTM encoder with a shared byte-pair encoding vocabulary for all languages, which is coupled with an auxiliary decoder and trained on publicly available parallel corpora. This enables us to learn a classifier on top of the resulting embeddings using English annotated data only, and transfer it to any of the 93 languages without any modification. Our experiments in cross-lingual natural language inference (XNLI data set), cross-lingual document classification (MLDoc data set), and parallel corpus mining (BUCC data set) show the effectiveness of our approach. We also introduce a new test set of aligned sentences in 112 languages, and show that our sentence embeddings obtain strong results in multilingual similarity search even for low- resource languages. Our implementation, the pre-trained encoder, and the multilingual test set are available at https://github.com/facebookresearch/LASER .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,886
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
RCV1: A New Benchmark Collection for Text Categorization Research
Lewis, D., Yang, Y., Rose, T. G., & Li, F. (2004) · Goldsmiths (University of London) · doi:10.5555/1005332.1005345 · Cited by 2,503
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049
Self-citationProceedings of The 32nd International Conference on Machine Learning
Germain, M., Gregor, K., Murray, I., & Larochelle, H. (2015) · Cited by 2,020
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00065 · Cited by 1,747
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586
Parallel Data, Tools and Interfaces in OPUS
Tiedemann, J. (2012) · Language Resources and Evaluation · Cited by 1,421
Exploiting Similarities among Languages for Machine Translation
Mikolov, T., Le, Q. V., & Sutskever, I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.4168 · Cited by 1,346
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293
Universal Sentence Encoder for English
Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., St. John, R., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/d18-2029 · Cited by 1,277
Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267
Mariani, J. (2018) · HAL · Cited by 1,151
Semi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
Understanding Back-Translation at Scale
Edunov, S., Ott, M., Auli, M., & Grangier, D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1045 · Cited by 993
XNLI: Evaluating Cross-lingual Sentence Representations
Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman, S., Schwenk, H., & Stoyanov, V. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1269 · Cited by 915
Self-citationPhrase-Based & Neural Unsupervised Machine Translation
Lample, G., Ott, M., Conneau, A., Denoyer, L., & Ranzato, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1549 · Cited by 615
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
Artetxe, M., Labaka, G., & Agirre, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1073 · Cited by 555
Self-citationAchieving Human Parity on Automatic Chinese to English News Translation
Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J. H., Federmann, C., Huang, X., Junczys-Dowmunt, M., Lewis, W. D., Li, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05567 · Cited by 521
Inducing Crosslingual Distributed Representations of Words
Klementiev, A., Titov, I., & Bhattarai, B. (2012) · International Conference on Computational Linguistics · Cited by 389
XNLI: Evaluating Cross-lingual Sentence RepresentationsPreprint
Conneau, A., Lample, G., Rinott, R., Williams, A., Bowman, S. R., Schwenk, H., & Stoyanov, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05053 · Cited by 353
Self-citationBilingual Word Representations with Monolingual Quality in Mind
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing · doi:10.3115/v1/w15-1521 · Cited by 315
Unsupervised Statistical Machine Translation
Artetxe, M., Labaka, G., & Agirre, E. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1399 · Cited by 315
Self-citationBilBOWA: Fast Bilingual Distributed Representations without Word Alignments
Gouws, S., Bengio, Y., & Corrado, G. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.2455 · Cited by 285
Massively Multilingual Word Embeddings
Ammar, W., Mulcaire, G., Tsvetkov, Y., Lample, G., Dyer, C., & Smith, N. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.01925 · Cited by 263
Artetxe, M., Labaka, G., & Agirre, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11992 · Cited by 220
Self-citationWord Translation Without Parallel Data
Conneau, A., Lample, G., Ranzato, M., Denoyer, L., & Jeǵou, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04087 · Cited by 219
Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Subramanian, S., Trischler, A., Bengio, Y., & Pal, C. J. (2018) · arXiv · doi:10.48550/arxiv.1804.00079 · Cited by 207
Learning Joint Multilingual Sentence Representations with Neural Machine Translation
Schwenk, H., & Douze, M. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2619 · Cited by 197
Self-citationRapid Adaptation of Neural Machine Translation to New Languages
Neubig, G., & Hu, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1103 · Cited by 185
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings
Artetxe, M., & Schwenk, H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1309 · Cited by 175
Self-citationLearning Language Representations for Typology Prediction
Malaviya, C., Neubig, G., & Littell, P. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1268 · Cited by 121
Effective Parallel Corpus Mining using Bilingual Sentence Embeddings
Guo, M., Shen, Q., Yang, Y., Ge, H., Cer, D. M., Ábrego, G. H., Stevens, K., Constant, N., Sung, Y.-H., Strope, B. P., et al. (2018) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w18-6317 · Cited by 111
A survey of cross-lingual embedding models.
Ruder, S. (2017) · Cited by 107
Filtering and Mining Parallel Data in a Joint Multilingual Space
Schwenk, H. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2037 · Cited by 104
Self-citationEspaña-Bonet, C., Varga, A. C., Barrón‐Cedeño, A., & van Genabith, J. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2764273 · Cited by 73
Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora
Zweigenbaum, P., Sharoff, S., & Rapp, R. (2017) · Workshop on Building and Using Comparable Corpora (BUCC) · doi:10.18653/v1/w17-2512 · Cited by 67
Overview of the Third BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora
Zweigenbaum, P., Sharoff, S., & Rapp, R. (2018) · HAL · Cited by 41
Multilingual Seq2seq Training with Similarity Loss for Cross-Lingual Document Classification
Yu, K., Li, H., & Oğuz, B. (2018) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w18-3023 · Cited by 33
Proceedings of the 10th Workshop on Building and Using Comparable Corpora
Rapp, R., Sharoff, S., & Zweigenbaum, P. (2017) · doi:10.18653/v1/w17-25 · Cited by 21
Grégoire, F., & Langlais, P. (2017) · doi:10.18653/v1/w17-2509 · Cited by 20
Embedding Learning Through Multilingual Concept Induction
Dufter, P., Zhao, M., Schmitt, M., Fraser, A., & Schütze, H. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1141 · Cited by 20
Proceedings of The Third Workshop on Representation Learning for NLP
(2018) · doi:10.18653/v1/w18-30 · Cited by 20
Cited by
fairseq: A Fast, Extensible Toolkit for Sequence Modeling
Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., & Auli, M. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-4009 · Cited by 2,344
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586
How Multilingual is Multilingual BERT?
Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., & Choi, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.595 · Cited by 1,232
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation
Reimers, N., & Gurevych, I. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.365 · Cited by 816
Metadata
- Authors
- Artetxe, M., & Schwenk, H.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2019-11
- Volume
- 7
- Pages
- 597-610
- ISSN
- 2331-8422 (ISSN-L)
- 2307-387X
- OpenAlex
- W2973088264
- Cited by
- 791
- References
- 59
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.