The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Open accessAbstract
We present MCTest, a freely available set of stories and associated questions intended for research on the machine comprehension of text.Previous work on machine comprehension (e.g., semantic modeling) has made great strides, but primarily focuses either on limited-domain datasets, or on solving a more restricted goal (e.g., open-domain relation extraction).In contrast, MCTest requires machines to answer multiple-choice reading comprehension questions about fictional stories, directly tackling the high-level goal of open-domain machine comprehension.Reading comprehension can test advanced abilities such as causal reasoning and understanding the world, yet, by being multiple-choice, still provide a clear metric.By being fictional, the answer typically can be found only in the story itself.The stories and questions are also carefully limited to those a young child would understand, reducing the world knowledge that is required for the task.We present the scalable crowd-sourcing methods that allow us to cheaply construct a dataset of 500 stories and 2000 questions.By screening workers (with grammar tests) and stories (with grading), we have ensured that the data is the same quality as another set that we manually edited, but at one tenth the editing cost.By being open-domain, yet carefully restricted, we hope MCTest will serve to encourage research and provide a clear metric for advancement on the machine comprehension of text.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
Age-of-acquisition ratings for 30,000 English words
Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0210-4 · Cited by 1,217
The 2005 PASCAL Visual Object Classes Challenge
Everingham, M., Zisserman, A., Williams, C. K. I., Van Gool, L., Allan, M., Bishop, C. M., Chapelle, O., Dalal, N., Deselaers, T., Dorkó, G., et al. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_8 · Cited by 765
Learning to parse database queries using inductive logic programming
Zelle, J. M., & Mooney, R. J. (1996) · Cited by 642
The labor economics of paid crowdsourcing
Horton, J. J., & Chilton, L. B. (2010) · Proceedings of the 11th ACM conference on Electronic commerce · doi:10.1145/1807342.1807376 · Cited by 601
Using Multiple Clause Constructors in Inductive Logic Programming for Semantic Parsing
Tang, L. R., & Mooney, R. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44795-4_40 · Cited by 261
Deep Read: a reading comprehension system
Hirschman, L., Light, M., Breck, E., & Burger, J. D. (1999) · Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics - · doi:10.3115/1034678.1034731 · Cited by 219
Toward A Model Of Children''s Story Comprehension
Eugene Charniak, D. (1972) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 213
Learning context-dependent mappings from sentences to logical form
Zettlemoyer, L., & Collins, M. (2009) · doi:10.3115/1690219.1690283 · Cited by 181
Automatic Gap-fill Question Generation from Text Books
Agarwal, M., & Mannem, P. (2011) · Cited by 109
Cited by
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Teaching Machines to Read and Comprehend
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,521
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Know What You Don't Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · arXiv · doi:10.48550/arxiv.1806.03822 · Cited by 2,242
Metadata
- Authors
- Richardson, M., Burges, C. J. C., & Renshaw, E.
- Venue
- Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
- Published
- 2013
- Pages
- 193-203
- OpenAlex
- W2125436846
- Cited by
- 610
- References
- 13
- FWCI
- 17.78
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text. https://app.cochat.ai/papers/10.18653/v1/d13-1020
- The paper
- Richardson, M., Burges, C. J. C., & Renshaw, E. (2013). MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/d13-1020
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.