The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
RACE: Large-scale ReAding Comprehension Dataset From Examinations
Open accessAbstract
We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task.Collected from the English exams for middle and high school Chinese students in the age range between 12 to 18, RACE consists of near 28,000 passages and near 100,000 questions generated by human experts (English instructors), and covers a variety of topics which are carefully designed for evaluating the students' ability in understanding and reasoning.In particular, the proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models (43%) and the ceiling human performance (95%).We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The type of questions to be generated range from the typical short answer span questions to multiple spans for question generation from the text [42, 47, 106, 107, 112].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [112]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, to our knowledge, most of the existing datasets are not of direct relevance to teaching and learning, except for RACE (Lai, Xie, Liu, Yang, & Hovy, 2017) and LearningQ (Chen, Yang, Hauff, & Houben, 2018).”
From Assessment in the age of artificial intelligence (2022) · cites this paper as Lai, Xie, Liu, Yang, & Hovy, 2017
The cited source says what the claim attributes to it.
In the citing paper
“On RACE-h and RACE-m (Lai et al., 2017), Chinchilla greatly outperforms Gopher , improving accuracy by more than 10% in both cases-see Table 7.”
From Training Compute-Optimal Large Language Models (2022) · cites this paper as Lai et al., 2017
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We consider only the case of human-readable prompts to large models, which contrasts with methods such as prompt-tuning [16,20], which learns a continuous prompt embedding, and auto-prompting [33], which attempts to search for prompts from scratch, both of which require a training step.”
From Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation With Large Language Models (2022) · cites this paper as [16]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“We choose word embedding size d = 100 and use the 100-dimensional Glove word embedding (Pennington et al., 2014) as embedding initialization.”
MethodBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 2×
“Instead of evaluating the accuracy, researchers need to use F1 score, BLEU (Papineni et al., 2002) or ROUGE (Lin and Hovy, 2003) as metrics, which measure the overlap between the prediction and ground truth answers since the questions come without candidate spans.”
MethodSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 8×
“Articles of SQUAD, NEWSQA and MS MARCO come from Wikipedia, CNN news and the Bing search engine respectively.”
MethodResultInfluentialTeaching Machines to Read and Comprehend
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,521
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473 · cited 3×
“Articles of SQUAD, NEWSQA and MS MARCO come from Wikipedia, CNN news and the Bing search engine respectively.”
BackgroundMethodTriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D. S., & Zettlemoyer, L. (2017) · arXiv · doi:10.48550/arxiv.1705.03551 · Cited by 1,650 · cited 1×
“, 2016) and recently proposed TriviaQA (Joshi et al., 2017).”
Automatic evaluation of summaries using N-gram co-occurrence statistics
Lin, C.-Y., & Hovy, E. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073465 · Cited by 1,578 · cited 2×
“, 2002) or ROUGE (Lin and Hovy, 2003) as metrics, which measure the overlap between the prediction and ground truth answers since the”
BackgroundSelf-citationNewsQA: A Machine Comprehension Dataset
Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2623 · Cited by 706 · cited 11×
“Articles of SQUAD, NEWSQA and MS MARCO come from Wikipedia, CNN news and the Bing search engine respectively.”
BackgroundMethodResultInfluentialMCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C. J. C., & Renshaw, E. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 610 · cited 4×
“Although efforts have been made with a similar motivation, including the MCTest dataset created by (Richardson et al., 2013) (containing 500 passages and 2000 questions) and several others (Peñas et al.”
BackgroundMethodInfluentialA Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
Chen, D., Bolton, J., & Manning, C. D. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1223 · Cited by 491 · cited 11×
“However, both require limited reasoning ability (Chen et al., 2016).”
BackgroundMethodInfluential
Cited by
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
“RACE has significantly longer context than other popular reading comprehension datasets and the proportion of questions that requires reasoning is very large.”
MethodInfluentialXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
“The RACE dataset [17] contains near 100K questions taken from the English exams for middle and high school Chinese students in the age range between 12 to 18, with the answers generated by human experts.”
Self-citationLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
“One of the most compelling signs of these breakthroughs is the evolution of machine performance on a reading comprehension task designed for middle and high-school English exams in China, the RACE test (Lai et al., 2017): the paper that originally describes the task and formulates the modeling challenge reports then s…”
BackgroundMethodInfluentialThe Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
“, 2017), RACE (Lai et al., 2017), BioASQ (http://bioasq.”
BackgroundThe Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096
“, 2017), RACE (Lai et al., 2017), BioASQ (http://bioasq.”
BackgroundPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
“…In-context Reading Comprehension: DROP (Dua et al., 2019), CoQA (Reddy et al., 2018), QuAC (Choi et al., 2018), SQuADv2 (Rajpurkar et al., 2018), RACE (Lai et al., 2017) • SuperGLUE (Wang et al., 2019a) • Natural Language Inference (NLI): Adversarial NLI (Nie et al., 2019) Table 4 includes…”
BackgroundInfluentialKnow What You Don't Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1806.03822 · Cited by 2,242
“Finally, some datasets, like MCTest (Richard-son et al., 2013) and RACE (Lai et al., 2017), pose multiple choice questions, which can have a “none of the above” option.”
BackgroundMethodInfluentialKnow What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
“Finally, some datasets, like MCTest (Richard-son et al., 2013) and RACE (Lai et al., 2017), pose multiple choice questions, which can have a “none of the above” option.”
BackgroundMethodInfluentialNatural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069
“There are a number of reading comprehension benchmarks based on multiple choice tests (Mihaylov et al., 2018; Richardson et al., 2013; Lai et al., 2017).”
BackgroundPre-Training With Whole Word Masking for Chinese BERT
Cui, Y., Che, W., Liu, T., Qin, B., & Yang, Z. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3124365 · Cited by 1,181
“As we traverse several popular machine reading comprehension benchmarks, such as SQuAD [3], CoQA [4], QuAC [5], NaturalQuestions [6], RACE [7], we can see that most of the top-performing models are based on BERT and its variants [8], [9], [10], demonstrating that the pre-trained language models…”
MethodParameter-efficient fine-tuning of large-scale pre-trained language models
Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00626-4 · Cited by 1,191
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
“RACE [225]: The RACE is a reading comprehension dataset collected from English examinations in China, which”
InfluentialA Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
“RACE [225]: The RACE is a reading comprehension dataset collected from English examinations in China, which”
InfluentialMS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION
He, P., Liu, X., Gao, J., & Chen, W. (2021) · International Conference on Learning Representations · Cited by 2,445
LEGAL-BERT: The Muppets straight out of Law School
Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., & Androutsopoulos, I. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.261 · Cited by 994
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
Patient Knowledge Distillation for BERT Model Compression
Sun, S., Cheng, Y., Gan, Z., & Liu, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1441 · Cited by 631
“The dataset is further divided into RACE-M and RACE-H, containing exam questions for middle school and high school students.”
BackgroundMethodInfluentialJin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685
“While many large-scale annotated general domain QA datasets have been introduced (Ra-jpurkar et al., 2016; Lai et al., 2017; Koˇcisk`y Question: Do preoperative statins reduce atrial fibrillation after coronary artery bypass grafting?”
BackgroundMultitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612
MPNet: Masked and Permuted Pre-training for Language Understanding
Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.09297 · Cited by 514
“The ReAding Comprehension from Examinations (RACE) (Lai et al., 2017) four choices.”
MethodInfluentialJin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718
A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT
Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., He, L., et al. (2025) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02443-6 · Cited by 557
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Self-citationDocRED: A Large-Scale Document-Level Relation Extraction Dataset
Yao, Y., Ye, D., Li, P., Han, X., Lin, Y., Liu, Z., Liu, Z., Huang, L., Zhou, J., & Sun, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1074 · Cited by 550
“…Mitamura et al., 2015, 2017), fact extraction and verification (Thorne et al., 2018), reading comprehension (Nguyen et al., 2016; Joshi et al., 2017; Lai et al., 2017), sentiment classification (Pang and Lee, 2004; Pret-tenhofer and Stein, 2010), summarization (Nalla-pati et al., 2016) and machine…”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
UNIFIEDQA: Crossing Format Boundaries with a Single QA System
Khashabi, D., Min, S., Khot, T., Sabharwal, A., Tafjord, O., Clark, P., & Hajishirzi, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.171 · Cited by 514
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.557 · Cited by 557
KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning
Lin, B. Y., Chen, X., Chen, J., & Ren, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1282 · Cited by 452
“2) there are also models using multi-task learning to transfer knowledge from other reading comprehension datasets, such as RACE (Lai et al., 2017) and”
BackgroundCrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Nangia, N., Vania, C., Bhalerao, R., & Bowman, S. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.154 · Cited by 446
Overview of the Transformer-based Models for NLP Tasks
Gillioz, A., Casas, J., Mugellini, E., & Khaled, O. A. (2020) · Annals of Computer Science and Information Systems · doi:10.15439/2020f20 · Cited by 385
“RACE : Reading Comprehension From Examinations (RACE) [27] is a collection of English questions set aside to Chinese students from middle school up to high school.”
BackgroundA Robustly Optimized BERT Pre-training Approach with Post-training
Liu, Z., Lin, W., Shi, Y., & Zhao, J. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-84186-7_31 · Cited by 403
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02232 · Cited by 354
“Many datasets have been created for MRC from formally written and objective texts, e.g., Wikipedia (WikiReading (Hewlett et al., 2016), SQuAD (Rajpurkar et al., 2016, 2018), Wiki-Hop (Welbl et al., 2018), DRCD (Shao et al., 2018), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018)) news and other articles (CNN/Dai…”
BackgroundDeBERTa: Decoding-enhanced BERT with Disentangled Attention
He, P., Liu, X., Gao, J., & Chen, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03654 · Cited by 420
He, P., Gao, J., & Chen, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09543 · Cited by 389
“‚ RACE is a large-scale machine reading comprehension dataset collected from English examinations in China designed for middle school and high school students (Lai et al., 2017).”
InfluentialAssessment in the age of artificial intelligence
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100075 · Cited by 409
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1300 · Cited by 2,979
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1023 · Cited by 370
“, 2015), suffer from even lower percentage of such questions (12% and 2% respectively (Lai et al., 2017)).”
BackgroundMethodInfluentialPPT: Pre-trained Prompt Tuning for Few-shot Learning
Gu, Y., Han, X., Liu, Z., & Huang, M. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.576 · Cited by 335
Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00317 · Cited by 342
“Question answering can also be interpreted as an exercise in verifying the knowledge of experts by finding the answer to trivia questions that are carefully crafted by someone who already knows the answer such that exactly one answer is correct such as TriviaQA and Quizbowl/Jeopoardy! questions (Ferrucci et al., 2010;…”
BackgroundMethodSurvey on evaluation methods for dialogue systems
Deriu, J., Rodrigo, A., Otegi, A., Echegoyen, G., Rosset, S., Agirre, E., & Cieliebak, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09866-x · Cited by 215
“This is the case of the well known MC dataset RACE (Lai et al.”
MethodBERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1242 · Cited by 644
“Many datasets have been created for MRC from formally written and objective texts, e.g., Wikipedia (WikiReading (Hewlett et al., 2016), SQuAD (Rajpurkar et al., 2016, 2018), Wiki-Hop (Welbl et al., 2018), DRCD (Shao et al., 2018), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018)) news and other articles (CNN/Dai…”
BackgroundMeasuring Massive Multitask Language Understanding
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2009.03300 · Cited by 291
“…topics like grade school subjects for which models can already achieve strong performance (Clark et al., 2018; Khot et al., 2019; Mihaylov et al., 2018; Clark et al., 2019), or are focused on linguistic understanding in the form of reading comprehension (Lai et al., 2017; Richardson et al., 2013).”
BackgroundMethodDeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Dai, D., Deng, C., Zhao, C., Xu, R. X., Gao, H., Chen, D., Li, J., Zeng, W., Yu, X., Wu, Y., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.70 · Cited by 291
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
Huang, L., Le Bras, R., Bhagavatula, C., & Choi, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1243 · Cited by 270
“Specifically, we first fine-tune BERT on RACE or SWAG or both, and directly test on C OS - MOS to show the impact of knowledge transfer.”
BackgroundResultInfluentialETC: Encoding Long and Structured Inputs in Transformers
Ainslie, J., Ontañón, S., Alberti, C., Cvicek, V., Fisher, Z., Pham, P., Ravula, A., Sanghai, S., Wang, Q., & Yang, L. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.19 · Cited by 268
“Many variants of the original Transformer model (Vaswani et al., 2017) have been proposed for scaling up training (RoBERTa, Liu et al., 2019), the internal representation (ALBERT, Lan et al., 2019), or both (T5, Raffel et al., 2019), outper-forming BERT (Devlin et al., 2018) in tasks such as GLUE (Wang et al., 2018),…”
MethodEvaluating Models’ Local Decision Boundaries via Contrast Sets
Gardner, M., Artzi, Y., Basmov, V., Berant, J., Bogin, B., Chen, S., Dasigi, P., Dua, D., Elazar, Y., Gottumukkala, A., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.117 · Cited by 256
“(2019), Lin et al. (2019), and Khashabi et al.”
CLUE: A Chinese Language Understanding Evaluation Benchmark
Xu, L., Hu, H., Zhang, X., Li, L., Cao, C., Li, Y., Xu, Y., Sun, K., Yu, D., Yu, C., et al. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.419 · Cited by 252
How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
Kaushik, D., & Lipton, Z. C. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1546 · Cited by 228
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
He, W., Liu, K., Liu, J., Lyu, Y., Zhao, S., Xiao, X., Liu, Y., Wang, Y., Wu, H., She, Q., et al. (2018) · Proceedings of the Workshop on Machine Reading for Question Answering · doi:10.18653/v1/w18-2605 · Cited by 226
“Multiple-choice MRC (Lai et al., 2017) tests comprehension ability via option selection on examination data.”
BackgroundScalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
Feng, Y., Chen, X., Lin, B. Y., Wang, P., Yan, J., & Ren, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.99 · Cited by 207
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Fisch, A., Talmor, A., Jia, R., Seo, M., Choi, E., & Chen, D. (2019) · Proceedings of the 2nd Workshop on Machine Reading for Question Answering · doi:10.18653/v1/d19-5801 · Cited by 215
“Li et al. (2019) argues that XLNet shows superior performances on datasets with discrete reasoning, such as DROP and RACE.”
Atlas: Few-shot Learning with Retrieval Augmented Language Models
Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., & Grave, E. (2022) · arXiv · doi:10.48550/arxiv.2208.03299 · Cited by 213
“The last, which we call transfer learning, leverages additional training examples from other multiple-choice QA tasks provided by the MMLU authors, namely MCTest (Richardson, 2013), RACE (Lai et al., 2017), ARC (Clark et al.”
MethodInfluentialNeural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000074 · Cited by 209
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., & Toutanova, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10044 · Cited by 205
SG-Net: Syntax-Guided Machine Reading Comprehension
Zhang, Z., Wu, Y., Zhou, J., Duan, S., Zhao, H., & Wang, R. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6511 · Cited by 194
Generative AI in Higher Education: The ChatGPT Effect
Chan, C. K. Y., & Colloton, T. (2024) · Routledge · doi:10.4324/9781003459026 · Cited by 203
Retrospective Reader for Machine Reading Comprehension
Zhang, Z., Yang, J., & Zhao, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17705 · Cited by 190
Ham, T. J., Lee, Y., Seo, S. H., Kim, S., Choi, H., Jung, S. J., & Lee, J. W. (2021) · 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca52012.2021.00060 · Cited by 192
“0, and RACE dataset [47], which is a large-scale reading comprehension dataset from examinations.”
BackgroundInteractive and Visual Prompt Engineering for Ad-hoc Task Adaptation With Large Language Models
Strobelt, H., Webson, A., Sanh, V., Hoover, B., Beyer, J., Pfister, H., & Rush, A. M. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209479 · Cited by 204
“The RACE dataset [14] is a multiple-choice reading comprehension dataset built from English examinations used for benchmarking this task.”
Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
Lv, S., Guo, D., Xu, J., Tang, D., Duan, N., Gong, M., Shou, L., Jiang, D., Cao, G., & Hu, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6364 · Cited by 183
Muppet: Massive Multi-task Representations with Pre-Finetuning
Aghajanyan, A., Gupta, A., Shrivastava, A., Chen, X., Zettlemoyer, L., & Gupta, S. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.468 · Cited by 183
“The pre-finetuning scale is reported in terms of the number of datasets. upon prior state of the art methods for RTE (Ben-tivogli et al., 2009) and HellaSWAG (Zellers et al., 2019), as well as improve upon vanilla pre-trained representations for MNLI (Williams et al., 2018a), SQuAD (Rajpurkar et al., 2016a), BoolQ (Cla…”
BackgroundMethodInfluentialSPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Vu, T., Lester, B., Constant, N., Al-Rfou’, R., & Cer, D. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.346 · Cited by 170
Jiang, Z., Araki, J., Ding, H., & Neubig, G. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00407 · Cited by 177
“…(fictional stories (Richardson et al., 2013)), PIQA (physical (Bisk et al., 2020)), SIQA (social (Sap et al., 2019)), RACE (English comprehension (Lai et al., 2017)), QASC (science (Khot et al., 2020)), MT-test (mixed (Hendrycks et al., 2020)) Extractive SQuAD 1.1 (wikipedia (Rajpurkar et al.,…”
InfluentialTraining Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932
“• Reading comprehension on RACE (Lai et al., 2017) • Common sense understanding on HellaSwag (Zellers et al., 2019), PIQA (Bisk et al., 2020), Wino-grande (Sakaguchi et al., 2020), SIQA (Sap et al., 2019), BoolQ (Clark et al., 2019), and TruthfulQA (Lin et al., 2021).”
Transformers as Soft Reasoners over Language
Clark, P., Tafjord, O., & Richardson, K. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/537 · Cited by 176
Hierarchical Graph Network for Multi-hop Question Answering
Fang, Y., Sun, S., Gan, Z., Pillai, R., Wang, S., & Liu, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.710 · Cited by 176
BBQ: A hand-built bias benchmark for question answering
Parrish, A., Chen, A., Nangia, N., Padmakumar, V., Phang, J., Thompson, J., Htut, P. M., & Bowman, S. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.165 · Cited by 186
Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and ApproachesPreprint
Storks, S., Gao, Q., & Chai, J. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01172 · Cited by 304
“This was done, for example, by the authors of RACE (Lai et al., 2017) and SWAG.”
BackgroundMethodInfluentialMetaICL: Learning to Learn In Context
Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.201 · Cited by 169
Symbolic Discovery of Optimization Algorithms
Chen, X., Liang, C., Huang, D., Real, E., Wang, K., Liu, Y., Pham, H., Dong, X., Luong, T., Hsieh, C.-J., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.06675 · Cited by 303
Dialog State Tracking: A Neural Reading Comprehension Approach
Gao, S., Sethi, A., Agarwal, S., Chung, T., & Hakkani-Tur, D. (2019) · Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w19-5932 · Cited by 164
“generic and can be extended to multiple choice questions by employing bilinear product for different types (Lai et al., 2017) or to free-form text by employing seq-to-seq models (Sutskever et al.”
BackgroundTransformer models used for text-based question answering systems
Nassiri, K., & Akhloufi, M. (2023) · Applied Intelligence · doi:10.1007/s10489-022-04052-8 · Cited by 164
“This dataset is from Chinese English examinations for middle school and high school students, which are intended to measure reading comprehension [28].”
MethodEnd-to-End Transformer-Based Models in Textual-Based NLP
Rahali, A., & Akhloufi, M. A. (2023) · AI · doi:10.3390/ai4010004 · Cited by 164
QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
Rogers, A., Gardner, M., & Augenstein, I. (2022) · ACM Computing Surveys · doi:10.1145/3560260 · Cited by 327
“They are “probing” in the same sense as, for example, RACE [156]: the only difference is that the text is a dialogue script.”
BackgroundMethodInfluentialNeural Machine Reading Comprehension: Methods and Trends
Liu, S., Zhang, X., Zhang, S., Wang, H., & Zhang, W. (2019) · Applied Sciences · doi:10.3390/app9183698 · Cited by 158
A Survey of Natural Language Generation
Dong, C., Li, Y., Gong, H., Chen, M., Li, J., Shen, Y., & Yang, M. (2022) · ACM Computing Surveys · doi:10.1145/3554727 · Cited by 310
“RACE.”
BackgroundInfluentialMultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
Talmor, A., & Berant, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1485 · Cited by 206
“Sun et al. (2018) has recently shown this in a larger experiment for multi-choice questions, where they first fine-tuned BERT on RACE (Lai et al., 2017) and then fine-tuned on several smaller datasets.”
BackgroundEfficient and Robust Question Answering from Minimal Context over Documents
Min, S., Zhong, V., Socher, R., & Xiong, C. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1160 · Cited by 170
“In addition, they cover different domains, including Wikipedia (Rajpurkar et al., 2016; Joshi et al., 2017), news (Hermann et al., 2015; Trischler et al., 2016), fictional stories (Richardson et al., 2013; Koˇcisk`y et al., 2017), and textbooks (Lai et al., 2017; Xie et al., 2017).”
BackgroundQuoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
Dasigi, P., Liu, N. F., Marasović, A., Smith, N. A., & Gardner, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1606 · Cited by 176
“6 in EM test score) on Q UOREF compared to other reading comprehension benchmarks: SQuAD and RACE (see Yang et al., 2019).”
MethodInfluentialGoing Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer
Powalski, R., Borchmann, Ł., Jurkiewicz, D., Dwojak, T., Pietruszka, M., & Pałka, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86331-9_47 · Cited by 247
Evaluating Commonsense in Pre-Trained Language Models
Zhou, X., Zhang, Y., Cui, L., & Huang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6523 · Cited by 230
ReClor: A Reading Comprehension Dataset Requiring Logical ReasoningPreprint
Yu, W., Jiang, Z., Dong, Y., & Feng, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04326 · Cited by 380
Savelka, J., Agarwal, A., An, M., Bogart, C., & Sakr, M. (2023) · Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3568813.3600142 · Cited by 141
“apply InstructGPT [36] and Codex to OpenBookQA [31], StoryCloze [33], and RACE-m [23] data sets which focus on multi-hop reasoning, recall, and reading comprehension, reporting 77.”
MethodInfluentialJEC-QA: A Legal-Domain Question Answering Dataset
Zhong, H., Xiao, C., Tu, C., Zhang, T., Liu, Z., & Sun, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6519 · Cited by 168
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language ProcessingPreprint
Dai, Z., Lai, G., Yang, Y., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03236 · Cited by 286
“Specifically, such tasks include the GLUE benchmark for language understanding [29, 30, 31, 32, 33, 34, 35, 36, 37], 7 widely used text (sentiment / topic) classification tasks (IMDB, AD, DBpedia, Yelp-2, Yelp-5, Amazon-2, Amazon-5) [38], and the RACE reading comprehension dataset [39].”
MethodSelf-citationAn Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3606231 · Cited by 136
“Reading Comprehension: We assess the LLMs’ performance on the RACE dataset [35], which includes both middle and high-school-level reading comprehension tasks.”
LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
Jian, L., Cui, L., Liu, H., Huang, D., Wang, Y., & Zhang, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/501 · Cited by 438
“, 2016] contains 1,080 questions for students in elementary schools; RACE [Lai et al., 2017] collects passages and questions from the English exams for middle school and high school Chinese students in ages between 12 to 18.”
InfluentialSelf-citationConvBERT: Improving BERT with Span-based Dynamic ConvolutionPreprint
Jiang, Z., Yu, W., Zhou, D., Chen, Y., Feng, J., & Yan, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02496 · Cited by 213
“Language model pre-training has shown great power for improving many natural language processing tasks [63, 52, 51, 33].”
BackgroundAn Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.08747 · Cited by 135
“Reading Comprehension: We assess the LLMs’ performance on the RACE dataset [35], which includes both middle and high-school-level reading comprehension tasks.”
Mulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132
“The type of questions to be generated range from the typical short answer span questions to multiple spans for question generation from the text [42, 47, 106, 107, 112].”
BackgroundImproving Machine Reading Comprehension with General Reading Strategies
Sun, K., Yu, D., Yu, D., & Cardie, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1270 · Cited by 134
“, 2018) with our proposed strategies on the largest general domain multiple-choice machine reading comprehension dataset RACE (Lai et al., 2017) that are collected from language exams (Section 4.”
BackgroundMethodInfluentialArabic Offensive Language on Twitter: Analysis and ExperimentsPreprint
Mubarak, H., Rashed, A. A.-A., Darwish, K. M., Samih, Y., & Abdelalí, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.02192 · Cited by 190
“Although Robustly Optimized BERT (RoBERTa) embeddings perform better than (BERT large ) on GLUE (Wang et al., 2018), RACE (Lai et al., 2017), and SQuAD (Ra-jpurkar et al., 2016) tasks, pre-trained multilingual RoBERTa models are not available.”
Conversational question answering: a survey
Zaib, M., Zhang, W. E., Sheng, Q. Z., Mahmood, A., & Zhang, Y. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01744-y · Cited by 188
“The examples of such datasets include RACE [37], MCTest [78], and MCSript [59], – Descriptive answer datasets allow answers to be in any free-form text.”
BackgroundAutomatic question generation and answer assessment: a survey
Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021) · Research and Practice in Technology Enhanced Learning · doi:10.1186/s41039-021-00151-1 · Cited by 123
Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right
Holtzman, A., West, P., Shwartz, V., Choi, Y., & Zettlemoyer, L. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.564 · Cited by 127
“Furthermore, recent evidence suggests that language is approximately uniformly information dense (Levy, 2018; Levy and Jaeger, 2007; Jaeger, 2006).”
BackgroundBiomedical Question Answering: A Survey of Approaches and Challenges
Jin, Q., Yuan, Z. P., Xiong, G., Yu, Q., Ying, H., Tan, C., Chen, M., Huang, S., Liu, X., & Yu, S. (2022) · ACM Computing Surveys · doi:10.1145/3490238 · Cited by 203
“Many general domain QA datasets that are extracted from examinations have been introduced [89, 95, 145, 176].”
Background
Metadata
- Authors
- Lai, G., Xie, Q., Liu, H., Yang, Y., & Hovy, E.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Published
- 2017
- Pages
- 785-794
- OpenAlex
- W2606964149
- Cited by
- 1,044
- References
- 19
- Citations by year
- 2026: 37 · 2025: 75 · 2024: 86 · 2023: 145 · 2022: 136 · 2021: 216 · 2020: 169 · 2019: 138 · 2018: 38 · 2017: 3
- FWCI
- 42.47
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency (FA8750, FA8750-12-2-0342)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- reading comprehension, machine reading comprehension, benchmark dataset, English examinations, Chinese students
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.