IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding
Open accessThe DOI matches its OpenAlex record; 22 of the 22 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“This document separation is crucial as it is used in the BERT architecture to extract long contiguous sequences (Devlin et al., 2019).”
BackgroundMethodResultInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 2×
“Following the notable success of contextual pre-trained language methods (Peters et al., 2018; De-vlin et al., 2019), several benchmarks to gauge the progress of general-purpose NLP research, such as GLUE (Wang et al., 2018), SuperGLUE (Wang et al., 2019), and CLUE (Xu et al., 2020), have been…”
BackgroundInfluentialUnsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Kudo, T., & Richardson, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/d18-2012 · Cited by 2,869 · cited 2×
“Vocabulary For both the IndoBERT and IndoBERT-lite models, we utilize Sentence-Piece (Kudo and Richardson, 2018) with a byte pair encoding (BPE) tokenizer as the vocabulary generation method.”
Method(2018) · International Conference on Learning Representations · Cited by 1,621
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051
FastText.zip: Compressing text classification models
Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1612.03651 · Cited by 887 · cited 2×
“For the word-embeddings-based model, we use an existing fastText model trained on the Indonesian Common Crawl (CC-ID) dataset (Joulin et al., 2016; Grave et al., 2018).”
MethodCamemBERT: a Tasty French Language Model
Martin, L., Muller, B., Ortiz Suárez, P. J., Dupont, Y., Romary, L., de la Clergerie, É., Seddah, D., & Sagot, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.645 · Cited by 754
OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles
Lison, P., & Tiedemann, J. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3fi26b3nobqg · Cited by 656 · cited 1×
“1 GB mixed LINDAT/CLARIAH-CZ OpenSubtitles (Lison and Tiedemann, 2016) 105,061,204 25,255,662 664.”
CLUE: A Chinese Language Understanding Evaluation BenchmarkPreprint
Xu, L., Zhang, X., Li, L., Hu, H., Cao, C., Liu, W., Li, J., Li, Y., Sun, K., Xu, Y., et al. (2020) · International Conference on Computational Linguistics · doi:10.48550/arxiv.2004.05986 · Cited by 487 · cited 6×
“CLUE consists of eight diverse tasks, including single-sentence, sentence-pair, and machine reading comprehension tasks.”
BackgroundInfluentialAsynchronous pipelines for processing huge corpora on medium to low resource infrastructures
Suárez, P. O., Sagot, B., & Romary, L. (2019) · Publication Server of the Institute for German Language (Institute for German Language) · doi:10.14618/ids-pub-9021 · Cited by 227
JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages
Agić, Ž., & Vulić, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1310 · Cited by 199
Emotion Classification on Indonesian Twitter Dataset
Saputri, M. S., Mahendra, R., & Adriani, M. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629262 · Cited by 100 · cited 5×
“EmoT An emotion classification dataset collected from the social media platform Twitter (Saputri et al., 2018).”
BackgroundMethodInfluentialSelf-citationDesigning an Indonesian part of speech tagset and manually tagged Indonesian corpus
Dinakaramani, A., Fam, R., Luthfi, A., & Manurung, H. (2014) · International Conference on Asian Language Processing · doi:10.1109/ialp.2014.6973519 · Cited by 98 · cited 2×
“arianti et al.,2007), sentiment analysis dataset (Purwarianti and Crisdayanti, 2019), aspect-based sentiment analysis dataset (Ilmania et al.,2018;Azhar et al.,2019), part-ofspeech (POS) tag dataset (Dinakaramani et al., 2014;Hoesen and Purwarianti,2018), named entity recognition (NER) dataset (Hoesen and Purwarianti,…”
BackgroundIlmania, A., Abdurrahman, A., Cahyawijaya, S., & Purwarianti, A. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629181 · Cited by 71 · cited 3×
“, 2007), sentiment analysis dataset (Purwarianti and Crisdayanti, 2019), aspect-based sentiment analysis dataset (Ilmania et al., 2018; Azhar et al., 2019), part-ofspeech (POS) tag dataset (Dinakaramani et al.”
BackgroundInfluentialSelf-citationToward a Standardized and More Accurate Indonesian Part-of-Speech Tagging
Kurniawan, K., & Aji, A. F. (2018) · International Conference on Asian Language Processing · doi:10.1109/ialp.2018.8629236 · Cited by 30
Kutuzov, A., & Kuzmenko, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03135 · Cited by 20 · cited 1×
“In addition, this setting is also suitable for contextual pre-training models that leverage inflections to improve the sentence-level representations (Kutuzov and Kuzmenko, 2019).”
BackgroundImproving Joint Layer RNN based Keyphrase Extraction by Using Syntactical Features
Mahfuzh, M., Soleman, S., & Purwarianti, A. (2019) · International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA) · doi:10.1109/icaicta.2019.8904194 · Cited by 6 · cited 1×
“…dataset (Dinakaramani et al., 2014; Hoesen and Purwarianti, 2018), named entity recognition (NER) dataset (Hoesen and Pur-warianti, 2018), span extraction dataset (Mahfuzh et al., 2019; Septiandri and Sutiono, 2019; Fer-nando et al., 2019), and textual entailment dataset (Setya and Mahendra, 2018).”
BackgroundSelf-citationSeptiandri, A. A., & Sutiono, A. P. (2019) · arXiv (Cornell University) · Cited by 3
Cited by
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.45 · Cited by 682
Self-citationIndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP
Koto, F., Rahimi, A., Lau, J. H., & Baldwin, T. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.66 · Cited by 343
GPT is an effective tool for multilingual psychological text analysis
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & Van Bavel, J. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308950121 · Cited by 297
“This was a dataset of tweets labeled with one of five emotions—anger, joy, sadness, fear, and love—by two annotators, taken from a previous study (86).”
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00447 · Cited by 179
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
Kreutzer, J., Caswell, I., Wang, L., Wahab, A., van Esch, D., Ulzii-Orshikh, N., Tapo, A., Subramani, N., Sokolov, A., Sikasote, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.12028 · Cited by 179
Metadata
- Authors
- Wilie, B., Vincentio, K., Winata, G. I., Cahyawijaya, S., Li, X., Lim, Z. Y., Soleman, S., Mahendra, R., Fung, P., Bahar, S., & Purwarianti, A.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-09-11
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2009.05387
- OpenAlex
- W3086966320
- Cited by
- 187
- References
- 22
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.