The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Open accessAbstract
We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples.TriviaQA includes 95K questionanswer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions.We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers.We also present two baseline algorithms: a featurebased classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension.Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that Trivi-aQA is a challenging testbed that is worth significant future study. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While past benchmarks have measured the ability to retrieve information (Joshi et al., 2017;Yang et al., 2018;Thorne et al., 2018;Dinan et al., 2019;Fan et al., 2019;Mialon et al., 2023), most of these benchmarks focus on retrieving information that can be found easily, and hence have become saturated by recent language models.”
From BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents (2025) · cites this paper as Joshi et al., 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these benchmarks primarily feature single-hop questions, which typically do not require multi-step reasoning or iterative retrieval.”
From BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents (2026) · cites this paper as Joshi et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On TriviaQA (Joshi et al., 2017) we show results for both the filtered (previously used in retrieval and open-book work) and unfiltered set (previously used in large language model evaluations). In both cases, Chinchilla substantially out performs Gopher .”
From Training Compute-Optimal Large Language Models (2022) · cites this paper as Joshi et al., 2017
The cited source says what the claim attributes to it.
In the citing paper
“strongly outperform recent approaches that use specialised pre-training objectives on TriviaQA [24].”
From Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) · cites this paper as [24]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“One reason to focus on imitative falsehoods is that they are less likely to be covered by existing question-answering benchmarks (Clark et al., 2018; Kwiatkowski et al., 2019; Joshi et al., 2017; Hendrycks et al., 2020).”
From TruthfulQA: Measuring How Models Mimic Human Falsehoods (2022) · cites this paper as Joshi et al., 2017
Experimental — accuracy not yet measured
In the citing paper
“LLMs perform better on nearly all datasets, such as on NaturalQuestions [62], WebQuestions [10], and TriviaQA [56].”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,698
Semantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1160 · Cited by 1,593
Building Watson: An Overview of the DeepQA Project
Ferrucci, D. A., Brown, E. W., Chu‐Carroll, J., Fan, J., Gondek, D. C., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J. M., et al. (2010) · AI Magazine · doi:10.1609/aimag.v31i3.2303 · Cited by 1,523
Modeling Relations and Their Mentions without Labeled Text
Riedel, S., Yao, L., & McCallum, A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15939-8_10 · Cited by 1,333
Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., & Weld, D. S. (2011) · Cited by 878
Self-citation
Research works printed but not linked: 7
- Jordan Boyd-Graber, Brianna Satinoff, He He, and Hal Daumé III. 2012. Besting the quiz master: Crowdsourcing incremental classification games. In Proceedings of the 2012 Joint Con- ference on Empirical Methods in Natural Lan- guage Processing and Computational Natural Lan- guage Learning. Association for Computational Linguistics, Jeju Island, Korea, pages 1290-1301. http://www.aclweb.org/anthology/D12-1118.
- Danqi Chen, Jason Bolton, and Christopher D. Man- ning. 2016. A thorough examination of the cnn/daily mail reading comprehension task. In Pro- ceedings of the 54th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages 2358-2367. http://www.aclweb.org/anthology/P16-1223.
- Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gim- pel, and David McAllester. 2016. Who did what: A large-scale person-centered cloze dataset. In Pro- ceedings of the 2016 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2230-2235. https://aclweb.org/anthology/D16- 1241.
- Denis Paperno, Germán Kruszewski, Angeliki Lazari- dou, Ngoc Quan Pham, Raffaella Bernardi, San- dro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernandez. 2016. The lambada dataset: Word prediction requiring a broad discourse con- text. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). Association for Computa- tional Linguistics, Berlin, Germany, pages 1525- 1534. http://www.aclweb.org/anthology/P16-1144.
- Panupong Pasupat and Percy Liang. 2015. Com- positional semantic parsing on semi-structured ta- bles. In Proceedings of the 53rd Annual Meet- ing of the Association for Computational Lin- guistics and the 7th International Joint Confer- ence on Natural Language Processing of the Asian Federation of Natural Language Process- ing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. pages 1470-1480. http://aclweb.org/anthology/P/P15/P15-1142.pdf.
- Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2015. Machine comprehension with syntax, frames, and semantics. In Proceedings of the 53rd Annual Meeting of the Association for Compu- tational Linguistics and the 7th International Joint Conference on Natural Language Processing (Vol- ume 2: Short Papers). Association for Computa- tional Linguistics, Beijing, China, pages 700-706. http://www.aclweb.org/anthology/P15-2115.
- Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierar- chical attention networks for document classifica- tion. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Lin- guistics, San Diego, California, pages 1480-1489. http://www.aclweb.org/anthology/N16-1174.
Cited by
Sorted by citations
Items 501–600 of 1,088
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question AnsweringPreprint
Wang, Z., Ng, P., Ma, X., Nallapati, R. M., & Xiang, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08167 · Cited by 23
TutorialVQA: Question Answering Dataset for Tutorial VideosPreprint
Colas, A., Kim, S., Dernoncourt, F., Gupte, S., Wang, D. Z., & Kim, D. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.01046 · Cited by 23
Is Retriever Merely an Approximator of Reader?Preprint
Yang, S., & Seo, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10999 · Cited by 50
A Reasoning-Focused Legal Retrieval Benchmark
Zheng, L., Guha, N., Arifov, J., Zhang, S., Skreta, M., Manning, C. D., Henderson, P., & Ho, D. E. (2025) · doi:10.1145/3709025.3712219 · Cited by 56
“…discussed several limitations of existing general IR benchmarks: a skew towards web/search-engine style retrieval tasks, tasks with low lexical and syntactic variance be-tween queries and gold passages, tasks with short query lengths, and propose new datasets to address these challenges [15, 35].”
RuBQ 2.0: An Innovated Russian Question Answering Dataset
Rybin, I., Korablinov, V., Efimov, P., & Braslavski, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77385-4_32 · Cited by 23
Question Answering Infused Pre-training of General-Purpose Contextualized Representations
Jia, R., Lewis, M., & Zettlemoyer, L. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.59 · Cited by 29
Self-citationBaradaran, R., & Amirkhani, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.095 · Cited by 25
Deep learning-based approach for Arabic open domain question answering
Alsubhi, K., Jamal, A. T., & Alhothali, A. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.952 · Cited by 24
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Shon, S., Arora, S., Lin, C.-J., Pasad, A., Wu, F. F., Sharma, R. S., Wu, W.-L., Lee, H.-Y., Livescu, K., & Watanabe, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.496 · Cited by 40
FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
Mao, Y., Dong, X., Xu, W., Gao, Y., Wei, B., & Zhang, Y. (2024) · ACM Transactions on Information Systems · doi:10.1145/3676957 · Cited by 39
Self-Supervised Test-Time Learning for Reading Comprehension
Banerjee, P., Gokhale, T., & Baral, C. R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.95 · Cited by 30
Quizbowl: The Case for Incremental Question AnsweringPreprint
Rodríguez, P., Feng, S., Iyyer, M., He, H., & Boyd-Graber, J. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04792 · Cited by 47
A Survey on Neural Machine Reading ComprehensionPreprint
Qiu, B., Xu, C., Xu, J., & Sun, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.03824 · Cited by 30
Hsu, T.-Y., Liu, C.-L., & Lee, H.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09587 · Cited by 22
Learning to Rank in Generative Retrieval
Li, Y., Yang, N., Wang, L., Wei, F., & Li, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i8.28717 · Cited by 27
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Vashurin, R., Fadeeva, E., Vazhentsev, A., Rvanova, L., Vasilev, D., Tsvigun, A., Petrakov, S., Xing, R., Sadallah, A., Grishchenkov, K., et al. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00737 · Cited by 21
Can language representation models think in bets?
Tang, Z., & Kejriwal, M. (2023) · Royal Society Open Science · doi:10.1098/rsos.221585 · Cited by 23
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
Yuan, F., Shou, L., Bai, X., Gong, M., Liang, Y., Duan, N., Fu, Y. H., & Jiang, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.87 · Cited by 21
Cheng, H., Chang, M.-W., Lee, K., & Toutanova, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.501 · Cited by 20
SRLGRN: Semantic Role Labeling Graph Reasoning Network
Chen, Z., & Kordjamshidi, P. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.714 · Cited by 23
Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance
Su, D., Yu, T., & Fung, P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.275 · Cited by 26
RoR: Read-over-Read for Long Document Machine Reading Comprehension
Zhao, J., Bao, J., Wang, Y., Zhou, Y., Wu, Y., He, X., & Zhou, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.160 · Cited by 22
Block-Skim: Efficient Question Answering for Transformer
Guan, Y., Li, Z., Lin, Z., Zhu, Y., Leng, J., & Guo, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21316 · Cited by 27
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
de Jong, M., Zemlyanskiy, Y., Ainslie, J., FitzGerald, N., Sanghai, S. K., Sha, F., & Cohen, W. W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.732 · Cited by 38
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering
Ju, M., Yu, W., Zhao, T., Zhang, C., & Ye, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.13 · Cited by 39
Subword-augmented Embedding for Cloze Reading Comprehension
Zhang, Z. (2024) · TIB Data Manager · doi:10.57702/xl3y0nut · Cited by 21
Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question AnsweringPreprint
Banerjee, P., & Baral, C. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03101 · Cited by 22
Contextualized Representations Using Textual Encyclopedic KnowledgePreprint
Joshi, M., Lee, K., Luan, Y., & Toutanova, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12006 · Cited by 31
Self-citationCut to the Chase: A Context Zoom-in Network for Reading Comprehension
Indurthi, S. R., Yu, S., Back, S., & Cuayáhuitl, H. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1054 · Cited by 20
Automatic Question Answering for Medical MCQs: Can It Go Further than Information Retrieval?
Ha, L. A., & Yaneva, V. (2019) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-056-4_049 · Cited by 20
Guo, X., Yu, M., Gao, Y., Gan, C., Campbell, M. S., & Chang, S. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.624 · Cited by 27
Reasoning over Hybrid Chain for Table-and-Text Open Domain Question Answering
Zhong, W., Huang, J., Liu, Q., Zhou, M., Wang, J., Yin, J. P., & Duan, N. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/629 · Cited by 20
Document Gated Reader for Open-Domain Question Answering
Wang, B., Yao, T., Zhang, Q., Xu, J., Tian, Z., Liu, K., & Zhao, J. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331190 · Cited by 19
Hardalov, M., Koychev, I. K., & Nakov, P. (2019) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-056-4_053 · Cited by 24
Distant Supervision for Multi-Stage Fine-Tuning in Retrieval-Based Question Answering
Xie, Y., Yang, W., Tan, L., Xiong, K., Yuan, N. J., Huai, B., Li, M., & Lin, J. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380060 · Cited by 18
Frame-based Neural Network for Machine Reading Comprehension
Guo, S., Guan, Y., Tan, H., Li, R., & Li, X. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106889 · Cited by 20
Subword-augmented Embedding for Cloze Reading ComprehensionPreprint
Zhang, Z., Huang, Y., & Zhao, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09103 · Cited by 20
Cross-Lingual Transfer Learning for Question AnsweringPreprint
Lee, C., & Lee, H.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.06042 · Cited by 29
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering ModelsPreprint
Guo, M., Yang, Y., Cer, D. M., Shen, Q., & Constant, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02507 · Cited by 26
Mention Memory: incorporating textual knowledge into Transformers through entity mention attentionPreprint
de Jong, M., Zemlyanskiy, Y., FitzGerald, N., Sha, F., & Cohen, W. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06176 · Cited by 40
Question Answering Survey: Directions, Challenges, Datasets, Evaluation MatricesPreprint
Pandya, H. A., & Bhatt, B. (2021) · Preprints.org · doi:10.20944/preprints202112.0136.v1 · Cited by 20
Generating fact checking briefs
Fan, A., Piktus, A., Petroni, F., Wenzek, G., Saeidi, M., Vlachos, A., Bordes, A., & Riedel, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.580 · Cited by 42
ReasonBERT: Pre-trained to Reason with Distant Supervision
Deng, X., Su, Y., Lees, A., Wu, Y., Yu, C., & Sun, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.494 · Cited by 27
Cross-Domain Slot Filling as Machine Reading Comprehension: A New Perspective
Liu, J., Yu, M., Chen, Y., & Xu, J. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3140559 · Cited by 19
Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
Hong, W., Zhang, Z., Wang, J., & Zhao, H. Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.76 · Cited by 19
Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering
Chuang, Y.-S., Fang, W., Li, S.-W. D., Yih, W.-T., & Glass, J. R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.768 · Cited by 43
Selectively Answering Ambiguous Questions
Cole, J., Zhang, M., Gillick, D., Eisenschlos, J., Dhingra, B., & Eisenstein, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.35 · Cited by 19
Salemi, A., & Zamani, H. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657733 · Cited by 35
Liu, Q., Han, S., Cambria, E., Li, Y., & Kwok, K. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10345-6 · Cited by 21
Learning to break: Knowledge-enhanced reasoning in multi-agent debate system
Wang, H., Du, X., Yu, W., Chen, Q., Zhu, K., Chu, Z., Yan, L., & Guan, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2024.129063 · Cited by 36
STARC: Structured Annotations for Reading Comprehension
Berzak, Y., Malmaud, J., & Lévy, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.507 · Cited by 22
Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?
Tang, Y., Ng, H. T., & Tung, A. K. H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.283 · Cited by 32
A survey on complex factual question answering
Zhang, L., Zhang, J., Ke, X., Li, H., Huang, X., Shao, Z., Cao, S., & Lv, X. (2022) · AI Open · doi:10.1016/j.aiopen.2022.12.003 · Cited by 24
Multi-Mention Learning for Reading Comprehension with Neural CascadesPreprint
Swayamdipta, S., Parikh, A. P., & Kwiatkowski, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00894 · Cited by 23
Hindsight: Posterior-guided training of retrievers for improved open-ended generationPreprint
Paranjape, A., Khattab, O., Potts, C., Zaharia, M., & Manning, C. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07752 · Cited by 30
An Open-Source Data Contamination Report for Large Language Models
Li, Y., Guo, Y., Guerin, F., & Lin, C. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.30 · Cited by 19
Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models
Zhang, X., Zhang, Y., Xie, W., Li, M., Dai, Z., Long, D., Xie, P., Zhang, M., Li, W., & Zhang, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00866 · Cited by 225
“For single-modal retrieval tasks, we utilize datasets including MS-MARCO [47], NQ [26], HotpotQA [70], TriviaQA [24], SQuAD [52], FEVER [56], and AllNLI for SimCSE [14], selecting a total of 1 million entries.”
Benchmarking Machine Reading Comprehension: A Psychological Perspective
Sugawara, S., Stenetorp, P., & Aizawa, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.137 · Cited by 22
Single-dataset Experts for Multi-dataset Question Answering
Friedman, D., Dodge, B., & Chen, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.495 · Cited by 22
Retrieval-guided Counterfactual Generation for QA
Paranjape, B., Lamm, M., & Tenney, I. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.117 · Cited by 19
Integration of cognitive tasks into artificial general intelligence test for large models
Qu, Y., Wei, C., Du, P., Che, W., Zhang, C., Ouyang, W., Bian, Y., Xu, F., Hu, B., Du, K., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109550 · Cited by 20
CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks
Li, X., Dou, Z., Zhou, Y., & Liu, F. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657778 · Cited by 24
Effective Character-Augmented Word Embedding for Machine Reading Comprehension
Zhang, Z., Huang, Y., Zhu, P., & Zhao, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99495-6_3 · Cited by 18
Chen, G., Yang, J., & Gašević, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-23204-7_6 · Cited by 19
Towards Interpreting BERT for Reading Comprehension Based QA
Ramnath, S., Nema, P., Sahni, D., & Khapra, M. M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.261 · Cited by 28
BERT-CoQAC: BERT-Based Conversational Question Answering in Context
Zaib, M., Tran, D. H., Sagar, S., Mahmood, A., Zhang, W. E., & Sheng, Q. Z. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-0010-4_5 · Cited by 18
What’s in a Name? Answer Equivalence For Open-Domain Question Answering
Si, C., Zhao, C., & Boyd-Graber, J. L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.757 · Cited by 17
Mishra, B. D., Huang, L., Tandon, N., Yih, W.-T., & Clark, P. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06975 · Cited by 18
Learning Cross-Context Entity Representations from TextPreprint
Ling, J., FitzGerald, N., Shan, Z., Soares, L. B., Févry, T., Weiss, D., & Kwiatkowski, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03765 · Cited by 25
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 143
Huang, J., Wang, M., Cui, Y., Liu, J., Chen, L., Wang, T., Li, H., & Wu, J. (2024) · Applied Sciences · doi:10.3390/app142311014 · Cited by 17
RWKV: Reinventing RNNs for the Transformer Era
Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Grella, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.13048 · Cited by 17
“• TriviaQA (Joshi et al., 2017) A QA-IR dataset which is constituted of triples of questions, answers, supporting evidence, and independently collected evidence documents, with an average of six documents per question for reliable sources.”
BackgroundGenerating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation
Li, Y., Liu, G., Wang, C., & Yang, Y. (2024) · arXiv · doi:10.48550/arxiv.2406.19234 · Cited by 17
“We use two datasets commonly employed in RAG attack research: Natural Questions [24] and Trivia-QA [25], in accordance with existing works [7], [26], [27].”
Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation
Li, Y., Liu, G., Wang, C., & Yang, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889013 · Cited by 52
RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
Jiang, W., Subramanian, S., Graves, C., Alonso, G., Yazdanbakhsh, A., & Dadu, V. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731093 · Cited by 23
“…percent of database vectors, the total bytes to scan per query is approximately: Here, 𝑃 scan is determined by evaluating a set of sample queries and analyzing the relationship between 𝑃 scan and retrieval quality measured by recall, as a common practice for retrieval configuration tuning [2].”
Can Explanations Be Useful for Calibrating Black Box Models?
Ye, X., & Durrett, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.429 · Cited by 23
Compositional Generalization in Multilingual Semantic Parsing over Wikidata
Cui, R., Aralikatte, R., Lent, H., & Hershcovich, D. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00499 · Cited by 28
Transformers with Learnable Activation Functions
Fang, H., Lee, J.-U., Moosavi, N. S., & Gurevych, I. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.181 · Cited by 24
Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers
Tam, W., Liu, X., Ji, K., Xue, L., Liu, J., Li, T., Dong, Y., & Tang, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.874 · Cited by 33
HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems
Tan, J., Dou, Z., Wang, W., Wang, M., Chen, W., & Wen, J.-R. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714546 · Cited by 48
Improving Spoken Question Answering Using Contextualized Word Representation
Su, D., & Fung, P. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053979 · Cited by 16
SD-QA: Spoken Dialectal Question Answering for the Real World
Faisal, F., Keshava, S., Alam, M. M. I., & Anastasopoulos, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.281 · Cited by 26
Tevatron: An Efficient and Flexible Toolkit for Neural Retrieval
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591805 · Cited by 17
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering
Chen, W., Verga, P., de Jong, M., Wieting, J., & Cohen, W. W. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.117 · Cited by 28
Li, H., Zhang, X., Liu, Y., Zhang, Y., Wang, Q., Zhou, X., Liu, J., Wu, H. R., & Wang, H. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5828 · Cited by 16
NLQuAD: A Non-Factoid Long Question Answering Data Set
Soleimani, A., Monz, C., & Worring, M. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.106 · Cited by 17
Review Conversational Reading ComprehensionPreprint
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00821 · Cited by 24
Transformer-Based Models for Question Answering on COVID19Preprint
Ngai, H., Park, Y., Chen, J., & Parsapoor, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.11432 · Cited by 33
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Yin, D., Brahman, F., Ravichander, A., Chandu, K., Chang, K.-W., Choi, Y., & Lin, B. Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.670 · Cited by 17
Xu, S., Pang, L., Shen, H., & Cheng, X. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557388 · Cited by 19
How to Quantify the Degree of Explainability: Experiments and Practical Implications
Sovrano, F., & Vitali, F. (2022) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee55066.2022.9882574 · Cited by 16
Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
Ma, K., Cheng, H., Liu, X., Nyberg, E., & Gao, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.392 · Cited by 23
KILM: Knowledge Injection into Encoder-Decoder Language Models
Xu, Y., Namazifar, M., Hazarika, D., Padmakumar, A., Liu, Y., & Hakkani-Tür, D. Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.275 · Cited by 26
Generative Multi-hop Retrieval
Lee, H., Yang, S., Oh, H., & Seo, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.92 · Cited by 26
Feng, T. H., Denny, P., Wuensche, B., Luxton-Reilly, A., & Hooper, S. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636263 · Cited by 16
On Early Detection of Hallucinations in Factual Question Answering
Snyder, B., Moisescu, M., & Zafar, M. B. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671796 · Cited by 46
Let Me Know What to Ask: Interrogative-Word-Aware Question Generation
Kang, J., Roman, H. P. S., & Myaeng, S.-H. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5822 · Cited by 16
RuBQ: A Russian Dataset for Question Answering over Wikidata
Korablinov, V., & Braslavski, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62466-8_7 · Cited by 23
Metadata
- Authors
- Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L.
- Affiliations
- University of Washington (US)
- Allen Institute for Artificial Intelligence (US)
- Venue
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2017
- Pages
- 1601-1611
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2612431505
- Cited by
- 1,690
- References
- 21
- Citations by year
- 2026: 141 · 2025: 205 · 2024: 157 · 2023: 220 · 2022: 204 · 2021: 259 · 2020: 198 · 2019: 190 · 2018: 97 · 2017: 19
- FWCI
- 61.63
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- IEEE DataPort (cc-by)
- Funders
- Tencent
- Defense Advanced Research Projects Agency (FA8750-13-2-0019, FA8750)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- TriviaQA, reading comprehension, distant supervision, trivia questions, SQuAD
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
