The DOI matches its OpenAlex record; 52 of the 52 references it cites were found in a registry or an index.
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2×
“Human performance For XNLI, PAWS-X, and XQuAD, we obtain human performance estimates from the English datasets they are derived from, MNLI, PAWS-X, and SQuAD respectively (Nangia & Bowman, 2019; Zhang et al., 2019; Rajpurkar et al., 2016).”
MethodUnsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
(2018) · International Conference on Learning Representations · Cited by 1,621
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586
How Multilingual is Multilingual BERT?
Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351
Exploiting Similarities among Languages for Machine Translation
Mikolov, T., Le, Q. V., & Sutskever, I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.4168 · Cited by 1,346
chrF: character n-gram F-score for automatic MT evaluation
Popović, M. (2015) · Proceedings of the Tenth Workshop on Statistical Machine Translation · doi:10.18653/v1/w15-3049 · Cited by 1,220
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Artetxe, M., & Schwenk, H. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00288 · Cited by 791
Word translation without parallel data
Lample, G., Conneau, A., Ranzato, M., Denoyer, L., & Jeǵou, H. (2018) · International Conference on Learning Representations · Cited by 653
Improving Vector Space Word Representations Using Multilingual Correlation
Faruqui, M., & Dyer, C. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1049 · Cited by 620
On the Cross-lingual Transferability of Monolingual Representations
Artetxe, M., Ruder, S., & Yogatama, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.421 · Cited by 598
Self-citationPAWS: Paraphrase Adversaries from Word Scrambling
Zhang, Y., Baldridge, J., & He, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1131 · Cited by 596
A Survey of Cross-lingual Word Embedding Models
Ruder, S., Vulić, I., & Søgaard, A. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.30462 · Cited by 510
Self-citationLearning bilingual word embeddings with (almost) no bilingual data
Artetxe, M., Labaka, G., & Agirre, E. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1042 · Cited by 455
Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?
Manning, C. D. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-19400-9_14 · Cited by 408
Massively Multilingual Neural Machine Translation
Aharoni, R., Johnson, M., & Firat, O. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1388 · Cited by 402
Self-citationXNLI: Evaluating Cross-lingual Sentence RepresentationsPreprint
Conneau, A., Lample, G., Rinott, R., Williams, A., Bowman, S. R., Schwenk, H., & Stoyanov, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05053 · Cited by 394
Inducing Crosslingual Distributed Representations of Words
Klementiev, A., Titov, I., & Bhattarai, B. (2012) · International Conference on Computational Linguistics · Cited by 389
MLQA: Evaluating Cross-lingual Extractive Question Answering
Lewis, P., Oguz, B., Rinott, R., Riedel, S., & Schwenk, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.653 · Cited by 387
Choosing Transfer Languages for Cross-Lingual Learning
Lin, Y.-H., Chen, C.-Y., Lee, J., Li, Z., Zhang, Y., Xia, M., Rijhwani, S., He, J., Zhang, Z., Ma, X., et al. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1301 · Cited by 344
Self-citationClark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00317 · Cited by 342
Bilingual Word Representations with Monolingual Quality in Mind
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing · doi:10.3115/v1/w15-1521 · Cited by 315
Cross-lingual Name Tagging and Linking for 282 Languages
Pan, X., Zhang, B., May, J., Nothman, J., Knight, K., & Ji, H. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1178 · Cited by 312
Multi-Source Transfer of Delexicalized Dependency Parsers
McDonald, R., Petrov, S., & Hall, K. (2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 295
BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
Gouws, S., Bengio, Y., & Corrado, G. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.2455 · Cited by 285
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Arivazhagan, N., Bapna, A., Firat, O., Lepikhin, D., Johnson, M., Krikun, M., Chen, M. X., Cao, Y., Foster, G., Cherry, C., et al. (2019) · arXiv · doi:10.48550/arxiv.1907.05019 · Cited by 279
Self-citationPAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Yang, Y., Zhang, Y., Tar, C., & Baldridge, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1382 · Cited by 223
Guzmán, F., Chen, P.-J., Ott, M., Pino, J., Lample, G., Koehn, P., Chaudhary, V., & Ranzato, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1632 · Cited by 203
How Translation Alters Sentiment
Mohammad, S. M., Salameh, M., & Kiritchenko, S. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4787 · Cited by 199
Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging
Täckström, O., Das, D., Petrov, S., McDonald, R., & Nivre, J. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00205 · Cited by 183
Schuster, T., Ram, O., Barzilay, R., & Globerson, A. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1162 · Cited by 182
Glavaš, G., Litschko, R., Ruder, S., & Vulić, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1070 · Cited by 180
Self-citationEarth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction
Zhang, M., Liu, Y., Luan, H., & Sun, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1207 · Cited by 170
Massively Multilingual Transfer for NER
Rahimi, A., Li, Y., & Cohn, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1015 · Cited by 167
Translationese and Its Dialects
Koppel, M., & Ordan, N. (2011) · Cited by 148
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?
Vulić, I., Glavaš, G., Reichart, R., & Korhonen, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1449 · Cited by 125
MultiFiT: Efficient Multi-lingual Language Model Fine-tuning
Eisenschlos, J. M., Ruder, S., Czapla, P., Kadras, M., Gugger, S., & Howard, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1572 · Cited by 111
Self-citationZero-Shot Cross-lingual Classification Using Multilingual Neural Machine TranslationPreprint
Eriguchi, A., Johnson, M., Fırat, O., Kazawa, H., & Macherey, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.04686 · Cited by 90
Self-citationBilingual Sentiment Embeddings: Joint Projection of Sentiment Across Languages
Barnes, J., Klinger, R., & Walde, S. S. I. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1231 · Cited by 68
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark
Nangia, N., & Bowman, S. R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1449 · Cited by 68
Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora
Zweigenbaum, P., Sharoff, S., & Rapp, R. (2017) · Workshop on Building and Using Comparable Corpora (BUCC) · doi:10.18653/v1/w17-2512 · Cited by 67
Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation
Siddhant, A., Johnson, M., Tsai, H., Ari, N., Riesa, J., Bapna, A., Fırat, O., & Raman, K. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i05.6414 · Cited by 64
Self-citationHsu, T.-Y., Liu, C.-L., & Lee, H.-Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1607 · Cited by 61
Semi-supervised Training Data Generation for Multilingual Question Answering
Lee, K., Yoon, K., Park, S., & Hwang, S. W. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3x9aotr9z8ks · Cited by 27
Does ‘well-being’ translate on Twitter?
Smith, L. K., Giorgi, S., Solanki, R., Eichstaedt, J. C., Schwartz, H. A., Abdul-Mageed, M., Buffone, A. E. K., & Ungar, L. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1217 · Cited by 20
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.41 · Cited by 1,629
Self-citationLanguage-agnostic BERT Sentence Embedding
Feng, F., Yang, Y., Cer, D., Arivazhagan, N., & Wang, W. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.62 · Cited by 758
mT5: A massively multilingual pre-trained text-to-text transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al‐Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11934 · Cited by 594
Self-citationCodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536
Metadata
- Authors
- Hu, J., Ruder, S., Siddhant, A., Neubig, G., Fırat, O., & Johnson, M.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-03-24
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2003.11080
- OpenAlex
- W3013840636
- Cited by
- 278
- References
- 52
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.