XNLI: Evaluating Cross-lingual Sentence Representations
Open accessThe DOI matches its DataCite record; 66 of the 66 references it cites were found in a registry or an index.
Abstract
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models. These models are generally trained on data in a single language (usually English), and cannot be directly used beyond that language. Since collecting data in every language is not realistic, there has been a growing interest in cross-lingual language understanding (XLU) and low-resource cross-language transfer. In this work, we construct an evaluation set for XLU by extending the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 15 languages, including low-resource languages such as Swahili and Urdu. We hope that our dataset, dubbed XNLI, will catalyze research in cross-lingual sentence understanding by providing an informative standard evaluation task. In addition, we provide several baselines for multilingual sentence understanding, including two based on machine translation systems, and two that use parallel data to train aligned multilingual bag-of-words and LSTM encoders. We find that XNLI represents a practical and challenging evaluation suite, and that directly translating the test data yields the best performance among available baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We evaluate two types of cross-lingual sentence encoders: (i) pretrained universal multilingual sentence embeddings based on the average of word embeddings (X-CBOW), (ii) bidirectionalLSTM (BiLSTM) (Hochreiter and Schmidhuber, 1997) sentence encoders trained on the MultiNLI training data (X-BILSTM).”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
(2014) · Cited by 33,419
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · cited 3×
“Recent crowdsourced annotation efforts have yielded datasets for NLI in English (Bowman et al., 2015; Williams et al., 2017) with nearly a million examples, and these have been widely used to evaluate neural network architectures and training strategies (Rocktäschel et al.”
MethodResultSelf-citationA Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Self-citation(2005) · Cited by 3,125
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1070 · Cited by 2,049 · cited 8×
“In that case, we consider our multilingual sentence embeddings as being pretrained and only learn a classifier on top of them to evaluate their quality, similar to socalled “transfer” tasks in (Kiros et al., 2015; Conneau et al., 2017) but in the multilingual setting.”
BackgroundResultInfluentialSelf-citationGoogle’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00065 · Cited by 1,747
(2018) · International Conference on Learning Representations · Cited by 1,621
Self-citationParallel Data, Tools and Interfaces in OPUS
Tiedemann, J. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/59wvhpiwbv7y · Cited by 1,599
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation
Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., & Specia, L. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2001 · Cited by 1,562 · cited 1×
“In addition to the Reuters corpus, Cer et al. (2017) propose sentence-level multilingual training and evaluation datasets for semantic textual similarity in four languages.”
BackgroundParallel Data, Tools and Interfaces in OPUS
Tiedemann, J. (2012) · Language Resources and Evaluation · Cited by 1,421
Exploiting Similarities among Languages for Machine Translation
Mikolov, T., Le, Q. V., & Sutskever, I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.4168 · Cited by 1,346
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293
Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267 · cited 2×
“…neural network architectures and training strategies (Rocktäschel et al., 2016; Gong et al., 2018; Peters et al., 2018; Wang et al., 2018), as well as to train effective, reusable sentence representations (Conneau et al., 2017; Subramanian et al., 2018; Cer et al., 2018; Conneau et al., 2018a).”
Background(2017) · International Conference on Learning Representations · Cited by 1,075
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907
Self-citationWsabie: Scaling up to large vocabulary image annotation
Weston, J., Samy Bengio, A., & Usunier, N. (2011) · Cited by 677
OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles
Lison, P., & Tiedemann, J. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3fi26b3nobqg · Cited by 656
Word translation without parallel data
Lample, G., Conneau, A., Ranzato, M., Denoyer, L., & Jeǵou, H. (2018) · International Conference on Learning Representations · Cited by 653
Self-citationUnsupervised Neural Machine Translation
Artetxe, M., Labaka, G., Agirre, E., & Cho, K. (2017) · arXiv · doi:10.48550/arxiv.1710.11041 · Cited by 632
Improving Vector Space Word Representations Using Multilingual Correlation
Faruqui, M., & Dyer, C. (2014) · Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1049 · Cited by 620
Phrase-Based & Neural Unsupervised Machine Translation
Lample, G., Ott, M., Conneau, A., Denoyer, L., & Ranzato, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1549 · Cited by 615
Self-citationReasoning about Entailment with Neural Attention
Rocktäschel, T., Grefenstette, E., Hermann, K. M., Kočiský, T., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06664 · Cited by 582
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2018) · arXiv · doi:10.48550/arxiv.1804.07461 · Cited by 562
Self-citationHypothesis Only Baselines in Natural Language Inference
Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., & Van Durme, B. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2023 · Cited by 480
SentEval: An Evaluation Toolkit for Universal Sentence Representations
Conneau, A., & Kiela, D. (2018) · arXiv · doi:10.48550/arxiv.1803.05449 · Cited by 477 · cited 1×
“The first approach is commonly used as a strong baseline for monolingual sentence embeddings (Arora et al., 2017; Conneau and Kiela, 2018; Gouews et al., 2014).”
BackgroundSelf-citationLearning bilingual word embeddings with (almost) no bilingual data
Artetxe, M., Labaka, G., & Agirre, E. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1042 · Cited by 455
SentEval: An Evaluation Toolkit for Universal Sentence Representations
Conneau, A., & Kiela, D. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2gscircifffd · Cited by 449
Self-citationInducing Crosslingual Distributed Representations of Words
Klementiev, A., Titov, I., & Bhattarai, B. (2012) · International Conference on Computational Linguistics · Cited by 389
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C. E., & Manning, C. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.05326 · Cited by 381
Self-citationNormalized Word Embedding and Orthogonal Transform for Bilingual Word Translation
Xing, C., Wang, D., Liu, C., & Lin, Y. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1104 · Cited by 381
The United Nations Parallel Corpus v1.0
Ziemski, M., Junczys-Dowmunt, M., & Pouliquen, B. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/263tmvnpzevv · Cited by 296
Optimizing Chinese word segmentation for machine translation performance
Chang, P.-C., Galley, M., & Manning, C. D. (2008) · Proceedings of the Third Workshop on Statistical Machine Translation - StatMT '08 · doi:10.3115/1626394.1626430 · Cited by 296
BilBOWA: Fast Bilingual Distributed Representations without Word Alignments
Gouws, S., Bengio, Y., & Corrado, G. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.2455 · Cited by 285
Multilingual Models for Compositional Distributed Semantics
Hermann, K. M., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1006 · Cited by 268
Massively Multilingual Word Embeddings
Ammar, W., Mulcaire, G., Tsvetkov, Y., Lample, G., Dyer, C., & Smith, N. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.01925 · Cited by 263
Self-citationWord Translation Without Parallel Data
Conneau, A., Lample, G., Ranzato, M., Denoyer, L., & Jeǵou, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04087 · Cited by 219
Self-citationNatural Language Inference over Interaction Space
Gong, Y., Luo, H., & Zhang, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.04348 · Cited by 208
Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning
Subramanian, S., Trischler, A., Bengio, Y., & Pal, C. J. (2018) · arXiv · doi:10.48550/arxiv.1804.00079 · Cited by 207
How Translation Alters Sentiment
Mohammad, S. M., Salameh, M., & Kiritchenko, S. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4787 · Cited by 199
Learning Joint Multilingual Sentence Representations with Neural Machine Translation
Schwenk, H., & Douze, M. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2619 · Cited by 197
Self-citationPerformance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment
Tsuchiya, M. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/26mjmiipm2qf · Cited by 194
Learning Word Vectors for 157 Languages
Grave, É., Bojanowski, P., Gupta, P., Joulin, A., & Mikolov, T. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2dwkd5xar3i5 · Cited by 187
Weston, J., Bengio, S., & Usunier, N. (n.d.) · Cited by 172
A Corpus for Multilingual Document Classification in Eight Languages
Schwenk, H., & Li, X. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/52gbaquzq5uu · Cited by 159
Self-citationThe IIT Bombay English-Hindi Parallel Corpus
Kunchukuttan, A., Mehta, P., & Bhattacharyya, P. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2erd8c5iqpzz · Cited by 147
Cross-Lingual Sentiment Classification with Bilingual Document Representation Learning
Zhou, X., Wan, X., & Xiao, J. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1133 · Cited by 128
Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot TranslationPreprint
Johnson, M., Schuster, M., Le, Q. V., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04558 · Cited by 109
Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings
Zhang, Y., Gaddy, D., Barzilay, R., & Jaakkola, T. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1156 · Cited by 107
Towards Universal Paraphrastic Sentence Embeddings
Wieting, J., Bansal, M., Gimpel, K., & Livescu, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08198 · Cited by 103
Multilingual Models for Compositional Distributed SemanticsPreprint
Hermann, K. M., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.4641 · Cited by 103
Distributed Word Representation Learning for Cross-Lingual Dependency Parsing
Xiao, M., & Guo, Y. (2014) · Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1613 · Cited by 98
Divide and Conquer: Crowdsourcing the Creation of Cross-Lingual Textual Entailment Corpora
Negri, M., Bentivogli, L., Mehdad, Y., & Giampiccolo, D. (n.d.) · Cited by 90
Learning Bilingual Word Representations by Marginalizing Alignments
Kočiský, T., Hermann, K. M., & Blunsom, P. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-2037 · Cited by 81
España-Bonet, C., Varga, A. C., Barrón‐Cedeño, A., & van Genabith, J. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2764273 · Cited by 73
Learning Distributed Representations for Multilingual Text Sequences
Pham, H., Luong, M.-T., & Manning, C. D. (2015) · Workshop on Vector Space Modeling for Natural Language Processing (VSM-NLP) · doi:10.3115/v1/w15-1512 · Cited by 63
Learning Bilingual Word Representations by Marginalizing AlignmentsPreprint
Kočiský, T., Hermann, K. M., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.0947 · Cited by 57
Using Bilingual Parallel Corpora for Cross-Lingual Textual Entailment
Mehdad, Y., Negri, M., & Federico, M. (2011) · Cited by 53
Baselines and Test Data for Cross-Lingual Inference
Agić, Ż., & Schluter, N. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2uttkx7m22xy · Cited by 21
Does ‘well-being’ translate on Twitter?
Smith, L. K., Giorgi, S., Solanki, R., Eichstaedt, J. C., Schwartz, H. A., Abdul-Mageed, M., Buffone, A. E. K., & Ungar, L. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1217 · Cited by 20
Research works printed but not linked: 11
- Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural ma- chine translation. In ICLR.
- Sarath Chandar, Mitesh M. Khapra, Balaraman Ravin- dran, Vikas Raykar, and Amrita Saha. 2013. Multi- lingual deep learning. In NIPS, Workshop Track.
- Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018a. What you can cram into a single vector: Probing sentence embeddings for linguistic properties. In ACL.
- Manaal Faruqui and Chris Dyer. 2014. Improving vec- tor space word representations using monolingual correlation. In EACL.
- Karl Moritz Hermann and Phil Blunsom. 2014. Multi- lingual models for compositional distributed seman- tics. In ACL, pages 58-68.
- Jeremy Howard and Sebastian Ruder. 2018. Fine-tuned language models for text classification. In ACL.
- Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018a. Unsupervised machine translation using monolingual corpora only. In ICLR.
- Guillaume Lample, Myle Ott, Alexis Conneau, Lu- dovic Denoyer, and Marc'Aurelio Ranzato. 2018b. Phrase-based & neural unsupervised machine trans- lation. In EMNLP.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013b. Distributed representa- tions of words and phrases and their compositional- ity. In NIPS, pages 3111-3119.
- Saif M. Mohammad, Mohammad Salameh, and Svet- lana Kiritchenko. 2016. How translation alters sen- timent. J. Artif. Int. Res., 55(1):95-130.
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word repre- sentations. In NAACL, volume 1, pages 2227-2237.
Cited by
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586
“Recent developments in learning and evaluating cross-lingual sentence representations in many languages [12] aim at mitigating the English-centric bias and suggest that it is possible to build universal cross-lingual encoders that can encode any sentence into a shared embedding space.”
BackgroundMethodInfluentialSelf-citationMultilingual Denoising Pre-training for Neural Machine Translation
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., & Zettlemoyer, L. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00343 · Cited by 1,527
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
Artetxe, M., & Schwenk, H. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00288 · Cited by 791
Self-citation
Metadata
- Authors
- Conneau, A., Lample, G., Rinott, R., Williams, A., Bowman, S. R., Schwenk, H., & Stoyanov, V.
- Affiliations
- New York University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1809.05053
- OpenAlex
- W2953109491
- Cited by
- 399
- References
- 63
- Citations by year
- 2026: 40 · 2025: 54 · 2024: 54 · 2023: 32 · 2022: 33 · 2021: 78 · 2020: 72 · 2019: 33 · 2018: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- arxiv.org/licenses/nonexclusive-distrib/1.0/
- Locations
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Tencent
- Samsung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- XNLI, natural language inference, low-resource languages, cross-lingual transfer, evaluation dataset, parallel data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.