The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Open accessAbstract
We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples.TriviaQA includes 95K questionanswer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions.We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers.We also present two baseline algorithms: a featurebased classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension.Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that Trivi-aQA is a challenging testbed that is worth significant future study. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While past benchmarks have measured the ability to retrieve information (Joshi et al., 2017;Yang et al., 2018;Thorne et al., 2018;Dinan et al., 2019;Fan et al., 2019;Mialon et al., 2023), most of these benchmarks focus on retrieving information that can be found easily, and hence have become saturated by recent language models.”
From BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents (2025) · cites this paper as Joshi et al., 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these benchmarks primarily feature single-hop questions, which typically do not require multi-step reasoning or iterative retrieval.”
From BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents (2026) · cites this paper as Joshi et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On TriviaQA (Joshi et al., 2017) we show results for both the filtered (previously used in retrieval and open-book work) and unfiltered set (previously used in large language model evaluations). In both cases, Chinchilla substantially out performs Gopher .”
From Training Compute-Optimal Large Language Models (2022) · cites this paper as Joshi et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“strongly outperform recent approaches that use specialised pre-training objectives on TriviaQA [24].”
From Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) · cites this paper as [24]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“One reason to focus on imitative falsehoods is that they are less likely to be covered by existing question-answering benchmarks (Clark et al., 2018; Kwiatkowski et al., 2019; Joshi et al., 2017; Hendrycks et al., 2020).”
From TruthfulQA: Measuring How Models Mimic Human Falsehoods (2022) · cites this paper as Joshi et al., 2017
Experimental — accuracy not yet measured
In the citing paper
“LLMs perform better on nearly all datasets, such as on NaturalQuestions [62], WebQuestions [10], and TriviaQA [56].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,698
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1160 · Cited by 1,593
Building Watson: An Overview of the DeepQA Project
Ferrucci, D. A., Brown, E. W., Chu‐Carroll, J., Fan, J., Gondek, D. C., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J. M., et al. (2010) · AI Magazine · doi:10.1609/aimag.v31i3.2303 · Cited by 1,523
Modeling Relations and Their Mentions without Labeled Text
Riedel, S., Yao, L., & McCallum, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15939-8_10 · Cited by 1,333
Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., & Weld, D. S. (2011) · Cited by 878
Self-citation
Research works printed but not linked: 7
- Jordan Boyd-Graber, Brianna Satinoff, He He, and Hal Daumé III. 2012. Besting the quiz master: Crowdsourcing incremental classification games. In Proceedings of the 2012 Joint Con- ference on Empirical Methods in Natural Lan- guage Processing and Computational Natural Lan- guage Learning. Association for Computational Linguistics, Jeju Island, Korea, pages 1290-1301. http://www.aclweb.org/anthology/D12-1118.
- Danqi Chen, Jason Bolton, and Christopher D. Man- ning. 2016. A thorough examination of the cnn/daily mail reading comprehension task. In Pro- ceedings of the 54th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages 2358-2367. http://www.aclweb.org/anthology/P16-1223.
- Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gim- pel, and David McAllester. 2016. Who did what: A large-scale person-centered cloze dataset. In Pro- ceedings of the 2016 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2230-2235. https://aclweb.org/anthology/D16- 1241.
- Denis Paperno, Germán Kruszewski, Angeliki Lazari- dou, Ngoc Quan Pham, Raffaella Bernardi, San- dro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez. 2016. The lambada dataset: Word prediction requiring a broad discourse con- text. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). Association for Computa- tional Linguistics, Berlin, Germany, pages 1525- 1534. http://www.aclweb.org/anthology/P16-1144.
- Panupong Pasupat and Percy Liang. 2015. Com- positional semantic parsing on semi-structured ta- bles. In Proceedings of the 53rd Annual Meet- ing of the Association for Computational Lin- guistics and the 7th International Joint Confer- ence on Natural Language Processing of the Asian Federation of Natural Language Process- ing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. pages 1470-1480. http://aclweb.org/anthology/P/P15/P15-1142.pdf.
- Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2015. Machine comprehension with syntax, frames, and semantics. In Proceedings of the 53rd Annual Meeting of the Association for Compu- tational Linguistics and the 7th International Joint Conference on Natural Language Processing (Vol- ume 2: Short Papers). Association for Computa- tional Linguistics, Beijing, China, pages 700-706. http://www.aclweb.org/anthology/P15-2115.
- Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierar- chical attention networks for document classifica- tion. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Lin- guistics, San Diego, California, pages 1480-1489. http://www.aclweb.org/anthology/N16-1174.
Cited by
Sorted by citations
Items 401–500 of 1,088
Varshney, N., Mishra, S. R., & Baral, C. R. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.158 · Cited by 54
Large Language Models are Built-in Autoregressive Search Engines
Ziems, N., Yu, W., Zhang, Z., & Jiang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.167 · Cited by 53
KI-MAG: A knowledge-infused abstractive question answering system in medical domain
Zafar, A., Sahoo, S. K., Bhardawaj, H., Das, A., & Ekbal, A. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127141 · Cited by 35
I Know There Is No Answer: Modeling Answer Validation for Machine Reading Comprehension
Tan, C., Wei, F., Zhou, Q. S., Yang, N., Lv, W., & Zhou, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99495-6_8 · Cited by 33
Interactive Language Learning by Question Answering
Yuan, X., Côté, M.-A., Fu, J., Lin, Z., Pal, C., Bengio, Y., & Trischler, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1280 · Cited by 36
Nguyen, K. V., Tran, K. V., Luu, S. T., Nguyen, A. G.-T., & Nguyen, N. L.-T. (2020) · IEEE Access · doi:10.1109/access.2020.3035701 · Cited by 32
Entity-Aware Abstractive Multi-Document Summarization
Zhou, H., Ren, W., Liu, G., Su, B., & Lu, W. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.30 · Cited by 33
Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks
Asai, A., Gardner, M., & Hajishirzi, H. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.162 · Cited by 47
Automatic Evaluation of Attribution by Large Language Models
Yue, X., Wang, B., Chen, Z., Zhang, K., Su, Y., & Sun, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.307 · Cited by 112
MixQG: Neural Question Generation with Mixed Answer Types
Murakhovs’ka, L., Wu, C.-S., Laban, P., Niu, T., Liu, W., & Xiong, C. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.111 · Cited by 31
Clickbait Spoiling via Question Answering and Passage Retrieval
Hagen, M., Fröbe, M., Jurk, A., & Potthast, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.484 · Cited by 35
Domain adaptation in reinforcement learning: a comprehensive and systematic study
Farhadi, A., Mirzarezaee, M., Sharifi, A., & Teshnehlab, M. (2024) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300668 · Cited by 35
Domain-agnostic Question-Answering with Adversarial Training
Lee, S., Kim, D., & Park, J. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5826 · Cited by 40
What Question Answering can Learn from Trivia Nerds
Boyd-Graber, J. L., & Börschinger, B. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.662 · Cited by 31
Harvesting and Refining Question-Answer Pairs for Unsupervised QA
Li, Z., Wang, W., Dong, L., Wei, F., & Xu, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.600 · Cited by 38
QuASE: Question-Answer Driven Sentence Encoding
He, H., Qiang, N., & Roth, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.772 · Cited by 31
Multilingual Transfer Learning for QA using Translation as Data Augmentation
Bornea, M. A., Pan, L., Rosenthal, S., Florian, R., & Sil, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17491 · Cited by 33
Dialogue Graph Modeling for Conversational Machine Reading
Ouyang, S., Zhang, Z., & Zhao, H. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.279 · Cited by 33
Aligning Generative Language Models with Human Values
Liu, R., Zhang, G., Feng, X., & Vosoughi, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.18 · Cited by 37
Nishida, K., Saito, I., Otsuka, A., Asano, H., & Tomita, J. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3271702 · Cited by 45
AI-based Question Answering Assistance for Analyzing Natural-language Requirements
Ezzini, S., Abualhaija, S., Arora, C., & Sabetzadeh, M. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00113 · Cited by 39
Chen, H.-T., Zhang, M., & Choi, E. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.146 · Cited by 31
Self-citationCoLT5: Faster Long-Range Transformers with Conditional Computation
Ainslie, J., Lei, T., de Jong, M., Ontañón, S., Brahma, S., Zemlyanskiy, Y., Uthus, D., Guo, M., Lee-Thorp, J., Tay, Y., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.309 · Cited by 38
A Multi-Stage Memory Augmented Neural Network for Machine Reading Comprehension
Yu, S., Indurthi, S. R., Back, S., & Lee, H. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2603 · Cited by 32
A Qualitative Comparison of CoQA, SQuAD 2.0 and QuACPreprint
Yatskar, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10735 · Cited by 46
Generation-Augmented Retrieval for Open-domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2020) · arXiv · doi:10.48550/arxiv.2009.08553 · Cited by 31
Chen, H.-T., Zhang, M. J. Q., & Choi, E. (2022) · arXiv · doi:10.48550/arxiv.2210.13701 · Cited by 31
Self-citationThree Bricks to Consolidate Watermarks for Large Language Models
Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., & Furon, T. (2023) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs58808.2023.10374576 · Cited by 63
Tracking Discrete and Continuous Entity State for Process Understanding
Gupta, A., & Durrett, G. (2019) · doi:10.18653/v1/w19-1502 · Cited by 34
Ni, J., Zhu, C., Chen, W., & McAuley, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1030 · Cited by 30
An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering
Longpre, S., Lu, Y., Tu, Z., & DuBois, C. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5829 · Cited by 40
Phrase Retrieval Learns Passage Retrieval, Too
Lee, J., Wettig, A., & Chen, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.297 · Cited by 41
MultiSpanQA: A Dataset for Multi-Span Question Answering
Li, H., Tomko, M., Vasardani, M., & Baldwin, T. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.90 · Cited by 33
Systematic Error Analysis of the Stanford Question Answering Dataset
Rondeau, M.-A., & Hazen, T. J. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2602 · Cited by 30
Multi-range Reasoning for Machine ComprehensionPreprint
Tay, Y., Luu, A. T., & Hui, S. (2018) · arXiv.org · doi:10.48550/arxiv.1803.09074 · Cited by 36
How Additional Knowledge can Improve Natural Language Commonsense Question Answering?Preprint
Mitra, A., Banerjee, P., Pal, K. K., Mishra, S. R., & Baral, C. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08855 · Cited by 57
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2311.12022 · Cited by 30
Large Language Models with Controllable Working Memory
Li, D., Rawat, A. S., Zaheer, M., Wang, X., Lukasik, M., Veit, A., Yu, F., & Kumar, S. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.112 · Cited by 46
Jing, Y., Xiong, D., & Yan, Z. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1249 · Cited by 29
Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering
Wang, Z., Ng, P., Nallapati, R. M., & Xiang, B. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.26 · Cited by 29
ViQuAE, a Dataset for Knowledge-based Visual Question Answering about Named Entities
Lerner, P. J., Ferret, O., Guinaudeau, C., Borgne, H. L., Besançon, R., Moreno, J. G., & Lovón-Melgarejo, J. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531753 · Cited by 39
Questions Are All You Need to Train a Dense Passage Retriever
Sachan, D. S., Lewis, M., Yogatama, D., Zettlemoyer, L., Pineau, J., & Zaheer, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00564 · Cited by 35
Self-citationLin, S.-C., Li, M., & Lin, J. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00556 · Cited by 35
Li, K., Xian, X., Wang, J., & Yu, N. (2018) · Applied Surface Science · doi:10.1016/j.apsusc.2018.11.214 · Cited by 30
Xu, K., Zhang, K., Li, J., Huang, W. D., & Wang, Y. (2024) · Electronics · doi:10.3390/electronics14010047 · Cited by 38
QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization
Yeh, Y. T., & Chen, Y.-N. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1333 · Cited by 33
On Making Reading Comprehension More Comprehensive
Gardner, M., Berant, J., Hajishirzi, H., Talmor, A., & Min, S. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5815 · Cited by 32
Chen, Q., Ji, F., Chen, H., & Zhang, Y. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.232 · Cited by 31
Reader-Guided Passage Reranking for Open-Domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.29 · Cited by 32
How Does Generative Retrieval Scale to Millions of Passages?
Pradeep, R., Hui, K., Gupta, J. P., Lelkes, A. D., Zhuang, H., Lin, J., Metzler, D., & Tran, V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.83 · Cited by 70
Pitfalls of Static Language Modelling.Preprint
Lazaridou, A., Kuncoro, A., Gribovskaya, E., Agrawal, D., Liska, A., Terzi, T., Giménez, M., d’Autume, C. D. M., Ruder, S., Yogatama, D., et al. (2021) · arXiv (Cornell University) · Cited by 32
Performance analysis of large language models in the domain of legal argument mining
Zubaer, A. A., Granitzer, M., & Mitrović, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1278796 · Cited by 37
Stochastic RAG: End-to-End Retrieval-Augmented Generation through Expected Utility Maximization
Zamani, H., & Bendersky, M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657923 · Cited by 41
From F to A on the New York Regents Science Exams — An Overview of the Aristo Project
Clark, P. E., Etzioni, O., Khashabi, D., Khot, T., Mishra, B. D., Richardson, K., Sabharwal, A., Schoenick, C., Tafjord, O., Tandon, N., et al. (2020) · AI Magazine · doi:10.1609/aimag.v41i4.5304 · Cited by 143
Thorne, J. P., Yazdani, M., Saeidi, M., Silvestri, F., Riedel, S., & Halevy, A. Y. (2021) · doi:10.18653/v1/2021.acl-long.241 · Cited by 36
xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering
Yang, N., Wei, F., Jiao, B., Jiang, D., & Yang, L. (2021) · doi:10.18653/v1/2021.acl-long.477 · Cited by 26
Semantic programming by example with pre-trained models
Verbruggen, G., Le, V., & Gulwani, S. (2021) · Proceedings of the ACM on Programming Languages · doi:10.1145/3485477 · Cited by 28
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints
Lu, A., Zhang, H., Zhang, Y., Wang, X., & Yang, D. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.148 · Cited by 52
A Thorough Examination on Zero-shot Dense Retrieval
Ren, R., Qu, Y., Liu, J., Zhao, X., Wu, Q., Ding, Y., Wu, H. Y., Wang, H., & Wen, J.-R. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1057 · Cited by 48
Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading ComprehensionPreprint
Nishida, K., Saito, I., Otsuka, A., Asano, H., & Tomita, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.10628 · Cited by 27
A Survey of Knowledge Enhanced Pre-trained ModelsPreprint
Yang, J., Xiao, G., Shen, Y.-L., Jiang, W., Hu, X., Zhang, Y., & Peng, J. (2021) · arXiv.org · doi:10.48550/arxiv.2110.00269 · Cited by 64
Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training
Lei, Y., Ding, L., Cao, Y. H., Zan, C., Yates, A., & Tao, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.695 · Cited by 42
Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering
Li, C., & Choi, J. D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.505 · Cited by 25
Challenges in Generalization in Open Domain Question Answering
Liu, L., Lewis, P., Riedel, S., & Stenetorp, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.155 · Cited by 34
ICDAR 2021 Competition on Document Visual Question Answering
Tito, R., Mathew, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86337-1_42 · Cited by 27
LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval
Xu, C., Guo, D., Duan, N., & McAuley, J. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.281 · Cited by 36
Transcending Scaling Laws with 0.1% Extra Compute
Tay, Y., Wei, J., Chung, H. W., Tran, V., So, D. R., Shakeri, S., Garcia, X., Zheng, S., Rao, J., Chowdhery, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.91 · Cited by 95
Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension
Mihaylov, T., & Frank, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1257 · Cited by 28
Conversations with Documents. An Exploration of Document-Centered AssistancePreprint
ter Hoeve, M., Sim, R., Nouri, E., Fourney, A., de Rijke, M., & White, R. W. (2020) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2002.00747 · Cited by 26
A Memory Efficient Baseline for Open Domain Question AnsweringPreprint
Izacard, G., Petroni, F., Hosseini, L., Cao, N. D., Riedel, S., & Grave, É. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15156 · Cited by 47
RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01352 · Cited by 26
Self-citationLogiqa 2.0—An improved dataset for logical reasoning in natural language understanding
Liu, H., Liu, J., Cui, L., Teng, Z., Duan, N., Zhou, M., & Zhang, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3293046 · Cited by 26
ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions
Parikh, S., Sai, A. B., Nema, P., & Khapra, M. M. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/594 · Cited by 25
Cao, Y. H., Fang, M., & Tao, D. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1032 · Cited by 25
Li, R., Jiang, Z., Wang, L., Lu, X., Zhao, M., & Chen, D. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106936 · Cited by 28
Knowing More About Questions Can Help: Improving Calibration in Question Answering
Zhang, S., Gong, C., & Choi, E. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.172 · Cited by 43
Self-citationReinforced History Backtracking for Conversational Question Answering
Qiu, M., Huang, X., Chen, C., Ji, F., Qu, C., Wei, W., Huang, J., & Zhang⋆, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17617 · Cited by 25
Contrastive Domain Adaptation for Question Answering using Limited Text Corpora
Yue, Z., Kratzwald, B., & Feuerriegel, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.754 · Cited by 29
Nonparametric Masked Language Modeling
Min, S., Shi, W., Lewis, M., Chen, X., Yih, W.-T., Hajishirzi, H., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.132 · Cited by 50
Self-citationSimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval
Zhou, K., Gong, Y., Liu, X., Zhao, W. X., Shen, Y., Dong, A., Lü, J., Majumder, R., Wen, J.-R., & Duan, N. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.56 · Cited by 29
HAIMA: A Hybrid SRAM and DRAM Accelerator-in-Memory Architecture for Transformer
Ding, Y., Liu, C., Duan, M., Chang, W., Li, K., & Li, K. (2023) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac56929.2023.10247913 · Cited by 25
Unifying Question Answering, Text Classification, and Regression via Span ExtractionPreprint
Keskar, N. S., McCann, B., Xiong, C., & Socher, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09286 · Cited by 50
Knowledge Enhanced Pretrained Language Models: A Compreshensive SurveyPreprint
Wei, X., Wang, S., Zhang, D., Bhatia, P., & Arnold, A. O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08455 · Cited by 57
DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments
Zheng, Y., Fu, D., Hu, X., Cai, X., Ye, L., Lu, P., & Liu, P. (2025) · arXiv · doi:10.48550/arxiv.2504.03160 · Cited by 25
“Specifically, we utilize NaturalQuestions (NQ) (Kwiatkowski et al., 2019) and TriviaQA (TQ) (Joshi et al., 2017) for single-hop scenarios, where answers can typically be found within a single web document.”
M-ANT: Efficient Low-bit Group Quantization for LLMs via Mathematically Adaptive Numerical Type
Hu, W., Zhang, H., Guo, C., Yu, F., Guan, R., Hua, Z., Liu, Z., Guan, Y., Guo, M., & Leng, J. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00086 · Cited by 32
“We use TruthfulQA [48] on lm-eval-harness for normal context evaluation and Trivi-aQA [36] on LongBench [4] for long context evaluation.”
Transformer-Based Extractive Social Media Question Answering on TweetQA
Butt, S., Ashraf, N., Fahim, H., Sidorov, G., & Gelbukh, A. F. (2021) · Computación y Sistemas · doi:10.13053/cys-25-1-3897 · Cited by 25
Peng, W., Hu, Y., Yu, J., Xing, L., & Xie, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107364 · Cited by 28
Uniqorn: Unified question answering over RDF knowledge graphs and natural language text
Pramanik, S., Alabi, J. O., Roy, R. S., & Weikum, G. (2024) · Journal of Web Semantics · doi:10.1016/j.websem.2024.100833 · Cited by 45
Dense Hierarchical Retrieval for Open-domain Question Answering
Liu, Y., Hashimoto, K., Zhou, Y., Yavuz, S., Xiong, C., & Yu, P. S. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.19 · Cited by 41
Explicit Contextual Semantics for Text ComprehensionPreprint
Zhang, Z., Wu, Y., Li, Z., & Zhao, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02794 · Cited by 33
InstAttention: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference
Pan, X., Li, E., Li, Q., Liang, S., Shan, Y., Zhou, K., Luo, Y., Wang, X., & Zhang, J. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00113 · Cited by 74
“The sequences for inference are sampled from popular datasets (i.e., ShareGPT [56], Wiki-Text-2 [45], SQuAD [59], and TriviaQA [27]).”
Huang, W., Zhou, G., Lapata, M., Vougiouklis, P., Montella, S., & Pan, J. Z. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113648 · Cited by 29
Structured Alignment Networks for Matching Sentences
Liu, Y., Gardner, M., & Lapata, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1184 · Cited by 25
Learning from On-Line User Feedback in Neural Question Answering on the Web
Kratzwald, B., & Feuerriegel, S. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313661 · Cited by 22
What’s Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering
Khot, T., Sabharwal, A., & Clark, P. E. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1281 · Cited by 24
CAiRE-COVID: A Question Answering and Multi-Document Summarization System for COVID-19 Research.
Su, D., Xu, Y., Yu, T., Siddique, F. B., Barezi, E. J., & Fung, P. (2020) · Europe PMC (PubMed Central) · Cited by 24
Iyer, S. V., Min, S., Mehdad, Y., & Yih, W.-T. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.100 · Cited by 25
Kaushik, D., Kiela, D., Lipton, Z. C., & Yih, W.-T. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.517 · Cited by 30
Metadata
- Authors
- Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L.
- Affiliations
- University of Washington (US)
- Allen Institute for Artificial Intelligence (US)
- Venue
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2017
- Pages
- 1601-1611
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2612431505
- Cited by
- 1,690
- References
- 21
- Citations by year
- 2026: 141 · 2025: 205 · 2024: 157 · 2023: 220 · 2022: 204 · 2021: 259 · 2020: 198 · 2019: 190 · 2018: 97 · 2017: 19
- FWCI
- 61.63
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- IEEE DataPort (cc-by)
- Funders
- Tencent
- Defense Advanced Research Projects Agency (FA8750-13-2-0019, FA8750)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- TriviaQA, reading comprehension, distant supervision, trivia questions, SQuAD
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
