HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Open accessAbstract
Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. We introduce HotpotQA, a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) we provide sentence-level supporting facts required for reasoning, allowing QA systems to reason with strong supervision and explain the predictions; (4) we offer a new type of factoid comparison questions to test QA systems' ability to extract relevant facts and perform necessary comparison. We show that HotpotQA is challenging for the latest QA systems, and the supporting facts enable models to improve performance and make explainable predictions.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Modifying the DSPy framework for joint optimization lowers cost while maintaining accuracy on HotPotQA.
“We modify the DSPy framework [24] for joint optimization, lowering cost while maintaining accuracy on HotPotQA [60].”
From AI Agents That Matter (2024) · cites this paper as [60]
Cites
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,171 · see citing works
Self-citationKnow What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240 · see citing works
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
MS MARCO: A Human Generated MAchine Reading COmprehension DatasetPreprint
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T. G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2025.5 · Cited by 1,190 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · arXiv · doi:10.48550/arxiv.1606.05250 · Cited by 678 · see citing works
Gated Self-Matching Networks for Reading Comprehension and Question Answering
Wang, W., Yang, N., Wei, F., Chang, B., & Zhou, M. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1018 · Cited by 669 · see citing works
The Web as a Knowledge-Base for Answering Complex Questions
Talmor, A., & Berant, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1059 · Cited by 520 · see citing works
Why We Need New Evaluation Metrics for NLG
Novikova, J., Dušek, O., Cercas Curry, A., & Rieser, V. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1238 · Cited by 422 · see citing works
Simple and Effective Multi-Paragraph Reading Comprehension
Clark, C., & Gardner, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1078 · Cited by 402 · see citing works
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
Dunn, M., Sagun, L., Higgins, M., Guney, V. U., Cirik, V., & Cho, K. (2017) · arXiv · doi:10.48550/arxiv.1704.05179 · Cited by 359 · see citing works
ParlAI: A Dialog Research Software Platform
Miller, A., Feng, W., Batra, D., Bordes, A., Fisch, A., Lu, J., Parikh, D., & Weston, J. (2017) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d17-2014 · Cited by 326 · see citing works
Stochastic Answer Networks for Machine Reading Comprehension
Liu, X., Shen, Y., Duh, K., & Gao, J. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1157 · Cited by 200 · see citing works
DCN+: Mixed Objective and Deep Residual Coattention for Question AnsweringPreprint
Xiong, C., Zhong, V., & Socher, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00106 · Cited by 90 · see citing works
MEMEN: Multi-layer Embedding with Memory Networks for Machine ComprehensionPreprint
Pan, B., Li, H., Zhao, Z., Cao, B., Cai, D., & He, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09098 · Cited by 58 · see citing works
Contextualized Word Representations for Reading Comprehension
Salant, S., & Berant, J. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2088 · Cited by 34 · see citing works
Mastering the Dungeon: Grounded Language Learning by Mechanical Turker DescentPreprint
Yang, Z., Zhang, S., Urbanek, J., Feng, W., Miller, A., Szlam, A. D., Kiela, D., & Weston, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07950 · Cited by 22 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
How is ChatGPT's behavior changing over time?
Chen, L., Zaharia, M., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2307.09009 · Cited by 507 · see citing works
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · see citing works
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876 · see citing works
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment AnalysisPreprint
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02232 · Cited by 504 · see citing works
Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., & Palomaki, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00317 · Cited by 432 · see citing works
♫ MuSiQue: Multihop Questions via Single-hop Question Composition
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00475 · Cited by 283 · see citing works
Autoregressive Entity RetrievalPreprint
Cao, N. D., Izacard, G., Riedel, S., & Petroni, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00904 · Cited by 307 · see citing works
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Xiong, L., Xiong, C., Li, Y., Tang, K.-F., Liu, J., Bennett, P., Ahmed, J., & Overwijk, A. (2020) · arXiv · doi:10.48550/arxiv.2007.00808 · Cited by 204 · see citing works
ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading ComprehensionPreprint
Zhang, S., Liu, X., Liu, J., Gao, J., Duh, K., & Durme, B. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.12885 · Cited by 201 · see citing works
Self-citationTransformers as Soft Reasoners over Language
Clark, P., Tafjord, O., & Richardson, K. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/537 · Cited by 176 · see citing works
The NLP Cookbook: Modern Recipes for Transformer Based Deep Learning Architectures
Singh, S. K., & Mahmood, A. (2021) · IEEE Access · doi:10.1109/access.2021.3077350 · Cited by 164 · see citing works
Hallucination‐Free? Assessing the Reliability of Leading <scp>AI</scp> Legal Research Tools
Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025) · Journal of Empirical Legal Studies · doi:10.1111/jels.12413 · Cited by 182 · see citing works
Self-citationLearning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1911.10470 · Cited by 152 · see citing works
Question Answering Over Temporal Knowledge Graphs
Saxena, A., Chakrabarti, S., & Talukdar, P. P. (2021) · doi:10.18653/v1/2021.acl-long.520 · Cited by 153 · see citing works
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems
Saad-Falcon, J., Khattab, O., Potts, C., & Zaharia, M. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.20 · Cited by 148 · see citing works
Training Question Answering Models From Synthetic Data
Puri, R., Spring, R., Shoeybi, M., Patwary, M. A., & Catanzaro, B. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.468 · Cited by 137 · see citing works
B reak It Down: A Question Understanding Benchmark
Wolfson, T., Geva, M., Gupta, A., Gardner, M., Goldberg, Y., Deutch, D., & Berant, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00309 · Cited by 136 · see citing works
Neural Module Networks for Reasoning over TextPreprint
Gupta, N., Lin, K., Roth, D., Singh, S., & Gardner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04971 · Cited by 94 · see citing works
Explaining Answers with Entailment Trees
Dalvi, B. B., Jansen, P. A., Tafjord, O., Xie, Z., Smith, H., Pipatanangkura, L., & Clark, P. E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.585 · Cited by 96 · see citing works
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03714 · Cited by 60 · see citing works
Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering
Banerjee, P., & Baral, C. R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.11 · Cited by 52 · see citing works
LooGLE: Can Long-Context Language Models Understand Long Contexts?
Li, J., Wang, M., Zheng, Z., & Zhang, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.859 · Cited by 55 · see citing works
Towards Continual Knowledge Learning of Language Models
Jang, J., Ye, S., Yang, S., Shin, J., Han, J., Kim, G., Choi, S. J., & Seo, M. (2021) · arXiv · doi:10.48550/arxiv.2110.03215 · Cited by 49 · see citing works
“Configuration KILT (Petroni et al., 2021) consists of 5 different tasks and 11 datasets: OpenDomain Question Answering (Joshi et al., 2017; Kwiatkowski et al., 2019; Fan et al., 2019; Yang et al., 2018), Fact Checking (Thorne et al., 2018), Entity Linking (Hoffart et al., 2011; Guo & Barbosa, 2018), Slot-filling (Levy…”
BackgroundDeep learning based question answering system in Bengali
Mayeesha, T. T., Sarwar, A. M., & Rahman, R. M. (2020) · Journal of Information and Telecommunication · doi:10.1080/24751839.2020.1833136 · Cited by 50 · see citing works
Esteva, A., Kale, A., Paulus, R., Hashimoto, K., Yin, W., Radev, D., & Socher, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09595 · Cited by 48 · see citing works
Machine Reading Comprehension: The Role of Contextualized Language Models and BeyondPreprint
Zhang, Z., Zhao, H., & Wang, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06249 · Cited by 54 · see citing works
Jhamtani, H., & Clark, P. E. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.10 · Cited by 43 · see citing works
Break It Down: A Question Understanding BenchmarkPreprint
Wolfson, T., Geva, M., Gupta, A., Gardner, M., Goldberg, Y., Deutch, D., & Berant, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11770 · Cited by 40 · see citing works
Understanding the planning of LLM agents: A survey
Huang, X., Liu, W., Chen, X., Wang, X., Wang, H., Lian, D., Wang, Y., Tang, R., & Chen, E. (2024) · arXiv · doi:10.48550/arxiv.2402.02716 · Cited by 47 · see citing works
Domain-agnostic Question-Answering with Adversarial Training
Lee, S., Kim, D., & Park, J. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5826 · Cited by 34 · see citing works
A Qualitative Comparison of CoQA, SQuAD 2.0 and QuACPreprint
Yatskar, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10735 · Cited by 31 · see citing works
A Survey on Explainability in Machine Reading ComprehensionPreprint
Thayaparan, M., Valentino, M., & Freitas, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00389 · Cited by 37 · see citing works
Hallucination Detection in Large Language Models with Metamorphic Relations
Yang, B., Mamun, M. A. A., Zhang, J. M., & Uddin, G. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715735 · Cited by 36 · see citing works
ETC: Encoding Long and Structured Inputs in TransformersPreprint
Ainslie, J., Ontañón, S., Alberti, C., Cvicek, V., Fisher, Z., Pham, P., Ravula, A., Sanghai, S. K., Wang, Q., & Li, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08483 · Cited by 27 · see citing works
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
Li, S., Li, X., Shang, L., Jiang, X., Liu, Q., Sun, C., Ji, Z., & Liu, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17568 · Cited by 23 · see citing works
FaVIQ: FAct Verification from Information-seeking Questions
Park, J., Min, S., Kang, J., Zettlemoyer, L., & Hajishirzi, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.354 · Cited by 28 · see citing works
Uniqorn: Unified question answering over RDF knowledge graphs and natural language text
Pramanik, S., Alabi, J. O., Roy, R. S., & Weikum, G. (2024) · Journal of Web Semantics · doi:10.1016/j.websem.2024.100833 · Cited by 32 · see citing works
On the Diversity and Limits of Human Explanations
Tan, C. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.158 · Cited by 27 · see citing works
Halterman, A., Keith, K. A., Sarwar, S. M., & O’Connor, B. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.371 · Cited by 20 · see citing works
Answering Any-hop Open-domain Questions with Iterative Document Reranking
Zhang, Y., Nie, P., Ramamurthy, A., & Song, L. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462853 · Cited by 25 · see citing works
Wu, Z., & Xu, H. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106075 · Cited by 19 · see citing works
Identifying Supporting Facts for Multi-hop Question Answering with Document Graph Networks
Thayaparan, M., Valentino, M., Schlegel, V., & Freitas, A. (2019) · Workshop on Graph Based Methods for Natural Language Processing · doi:10.18653/v1/d19-5306 · Cited by 16 · see citing works
PathReasoner: Explainable reasoning paths for commonsense question answering
Zhan, X., Huang, Y., Dong, X. M., Cao, Q., & Liang, X. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107612 · Cited by 18 · see citing works
Review Conversational Reading ComprehensionPreprint
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00821 · Cited by 17 · see citing works
ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Xu, B., Peng, Z., Lei, B., Mukherjee, S., Liu, Y., & Xu, D. (2023) · arXiv · doi:10.48550/arxiv.2305.18323 · Cited by 17 · see citing works
Neural Conversational QA: Learning to Reason vs Exploiting Patterns
Verma, N., Sharma, A., Madan, D., Contractor, D., Kumar, H., & Joshi, S. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.589 · Cited by 14 · see citing works
Liu, N. F., Lee, T., Jia, R., & Liang, P. (2021) · arXiv (Cornell University) · Cited by 17 · see citing works
Conditional Self-Attention for Query-based SummarizationPreprint
Xie, Y., Zhou, T., Mao, Y., & Chen, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.07338 · Cited by 17 · see citing works
Visuo-Linguistic Question Answering (VLQA) Challenge
Sampat, S. K., Yang, Y., & Baral, C. R. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.413 · Cited by 13 · see citing works
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural NetworkPreprint
Ye, D., Lin, Y., Liu, Z., Liu, Z., & Sun, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02170 · Cited by 13 · see citing works
Reference Knowledgeable Network for Machine Reading Comprehension
Zhao, Y., Zhang, Z., & Zhao, H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3164219 · Cited by 13 · see citing works
Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge.Preprint
Talmor, A., Tafjord, O., Clark, P. E., Goldberg, Y., & Berant, J. (2020) · arXiv (Cornell University) · Cited by 12 · see citing works
Conversational recommendation based on end-to-end learning: How far are we?
Manzoor, A., & Jannach, D. (2021) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2021.100139 · Cited by 11 · see citing works
ERNIE-Doc: A Retrospective Long-Document Modeling TransformerPreprint
Ding, S., Shang, J., Wang, S., Sun, Y., Tian, H., Wu, H. R., & Wang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15688 · Cited by 11 · see citing works
Bhakthavatsalam, S., Khashabi, D., Khot, T., Mishra, B. D., Richardson, K., Sabharwal, A., Schoenick, C., Tafjord, O., & Clark, P. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03315 · Cited by 11 · see citing works
Answer Complex Questions: Path Ranker Is All You Need
Zhang, X., Zhan, K., Hu, E., Fu, C., Luo, L., Jiang, H., Jia, Y., Yu, F., Dou, Z., Cao, Z., et al. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462942 · Cited by 10 · see citing works
Knowledge-Aided Open-Domain Question AnsweringPreprint
Zhou, M., Shi, Z., Huang, M., & Zhu, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05244 · Cited by 11 · see citing works
Robustifying Multi-hop QA through Pseudo-Evidentiality Training
Lee, K., Hwang, S.-W., Han, S.-E., & Lee, D. (2021) · doi:10.18653/v1/2021.acl-long.476 · Cited by 9 · see citing works
Directional attention weaving for text-grounded conversational question answering
Li, R., Jiang, Z., Wang, L., Lu, X., & Zhao, M. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.056 · Cited by 9 · see citing works
Multi-level retrieval with semantic Axiomatic Fuzzy Set clustering for question answering
Lang, Q., Liu, X., & Deng, Y. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107858 · Cited by 9 · see citing works
Addressing Semantic Drift in Question Generation for Semi-Supervised Question AnsweringPreprint
Zhang, S., & Bansal, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06356 · Cited by 14 · see citing works
Self-citationKapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. (2024) · arXiv · doi:10.48550/arxiv.2407.01502 · Cited by 13 · see citing works
Learning with Instance Bundles for Reading Comprehension
Dua, D., Dasigi, P., Singh, S., & Gardner, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.584 · Cited by 8 · see citing works
Interactive Language Learning by Question AnsweringPreprint
Yuan, X., Côté, M.-A., Fu, J., Lin, Z., Pal, C., Bengio, Y., & Trischler, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10909 · Cited by 11 · see citing works
Self-citationHuman Evaluation of Spoken vs. Visual Explanations for Open-Domain QA
Gonzalez, A. V., Bansal, G., Fan, A., Jia, R., Mehdad, Y., & Iyer, S. (2020) · arXiv · doi:10.48550/arxiv.2012.15075 · Cited by 7 · see citing works
Evaluating Explanations for Reading Comprehension with Realistic Counterfactuals.Preprint
Ye, X., Nair, R., & Durrett, G. (2021) · arXiv (Cornell University) · Cited by 6 · see citing works
SnapKV: LLM Knows What You are Looking for Before Generation
Li, Y., Huang, Y., Yang, B., Venkitesh, B., Locatelli, A., Ye, H., Cai, T., Lewis, P., & Chen, D. (2024) · arXiv · doi:10.48550/arxiv.2404.14469 · Cited by 74 · see citing works
Answering POI-recommendation Questions using Tourism Reviews
Contractor, D., Shah, K., Partap, A., Singla, P., & Mausam, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482320 · Cited by 6 · see citing works
Chia, Y. K., Witteveen, S., & Andrews, M. (2020) · doi:10.18653/v1/2020.textgraphs-1.14 · Cited by 5 · see citing works
Explainable Natural Language Reasoning via Conceptual Unification.Preprint
Valentino, M., Thayaparan, M., & Freitas, A. (2020) · arXiv (Cornell University) · Cited by 5 · see citing works
A Self-supervised Method for Entity AlignmentPreprint
Liu, X., Hong, H., Wang, X., Chen, Z., Kharlamov, E., Dong, Y., & Tang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09395 · Cited by 5 · see citing works
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
Zhang, Q., Wang, D., Qian, H., Li, Y., Zhang, T., Huang, M., Xu, K., Li, H., Yan, L., & Qiu, H. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1314 · Cited by 4 · see citing works
The Graph Reasoning Approach Based on the Dynamic Knowledge Auxiliary for Complex Fact Verification
Wang, Y., Xia, C., Si, C., Zhang, C., & Wang, T. (2020) · Electronics · doi:10.3390/electronics9091472 · Cited by 4 · see citing works
The TechQA DatasetPreprint
Castelli, V., Chakravarti, R., Dana, S., Ferritto, A., Florian, R., Franz, M., Garg, D., Khandelwal, D., McCarley, S., McCawley, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02984 · Cited by 12 · see citing works
Tang, H., Liu, J., Li, H., Yu, H., Wu, H. R., & Wang, H. (2020) · arXiv (Cornell University) · Cited by 3 · see citing works
Knollmeyer, S., Caymazer, O., Koval, L., Akmal, M. U., Asif, S., Mathias, S. G., & Großmann, D. (2026) · SN Computer Science · doi:10.1007/s42979-026-05134-x · Cited by 6 · see citing works
Empowering Large Language Model Agent through Step-Level Self-Critique and Self-Training
Zhai, Y., Liu, H., Zhang, Z., Lin, T., Xu, K., Yang, C., Feng, D., Ding, B., & Wang, H. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3729965 · Cited by 3 · see citing works
Toward conversational agents with context and time sensitive long-term memory
Alonso, N., Figliolia, T., Ndirango, A., & Millidge, B. (2024) · arXiv · doi:10.48550/arxiv.2406.00057 · Cited by 2 · see citing works
BioWorkflow: Retrieving comprehensive bioinformatics workflows from publications
Wang, Y., & Wang, J. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf571 · Cited by 2 · see citing works
Betz, G., & Richardson, K. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.900943 · Cited by 2 · see citing works
On the Transferability of Minimal Prediction Preserving Inputs in Question Answering
Longpre, S., Lu, Y., & DuBois, C. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.101 · Cited by 2 · see citing works
A Study on Forecasting Order Quantity from Manufacturing Supply Chain Data using XAI
Jung, Y., & Yoon, C. (2024) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2024.22.8.41 · Cited by 2 · see citing works
A Cognitive Method for Automatically Retrieving Complex Information on a Large Scale
Wang, Y., Yao, B., Wang, T., Xia, C., & Zhao, X. (2020) · Sensors · doi:10.3390/s20113057 · Cited by 2 · see citing works
Semantic Graphs for Generating Deep QuestionsPreprint
Pan, L., Xie, Y., Feng, Y., Chua, T., & Kan, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12704 · Cited by 2 · see citing works
VisBERT: Hidden-State Visualizations for TransformersPreprint
van Aken, B., Winter, B., Löser, A., & Gers, F. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04507 · Cited by 2 · see citing works
DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
Singhvi, A., Shetty, M., Tan, S., Potts, C., Sen, K., Zaharia, M., & Khattab, O. (2023) · arXiv · doi:10.48550/arxiv.2312.13382 · Cited by 2 · see citing works
Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
Krishna, S., Krishna, K., Mohananey, A., Schwarcz, S., Stambler, A., Upadhyay, S., & Faruqui, M. (2024) · arXiv · doi:10.48550/arxiv.2409.12941 · Cited by 1 · see citing works
EvoAgentX: An Automated Framework for Evolving Agentic Workflows
Wang, Y., Liu, S., Fang, J., & Meng, Z. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2025.emnlp-demos.47 · Cited by 1 · see citing works
A Cognitive Automation Approach for a Smart Lending and Early Warning Application.
Oro, E., Ruffolo, M., & Pupo, F. (2020) · EDBT/ICDT Workshops · Cited by 1 · see citing works
A Study of the Tasks and Models in Machine Reading ComprehensionPreprint
Wang, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08635 · Cited by 2 · see citing works
Asking Complex Questions with Multi-hop Answer-focused ReasoningPreprint
Ma, X., Zhu, Q., Zhou, Y., Li, X., & Wu, D. O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07402 · Cited by 1 · see citing works
Understanding in Artificial IntelligencePreprint
Maetschke, S., Iraola, D. M., Barnard, P., ShafieiBavani, E., Zhong, P., Xu, Y., & Yepes, A. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06573 · Cited by 2 · see citing works
Evaluating Document Coherence ModellingPreprint
Shen, A., Mistica, M., Salehi, B., Li, H., Baldwin, T., & Qi, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.10133 · Cited by 1 · see citing works
Improving Cross-Lingual Reading Comprehension with Self-TrainingPreprint
Huang, W., Huang, C., & Lee, H.-Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03627 · Cited by 1 · see citing works
Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive StudyPreprint
Mou, X., Yang, C., Yu, M., Yao, B., Guo, X., Potdar, S., & Su, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03826 · Cited by 1 · see citing works
Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?Preprint
Choudhury, S. R., Bhutani, N., & Augenstein, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07102 · Cited by 1 · see citing works
Representation Decoupling for Open-Domain Passage Retrieval.Preprint
Wu, B., Zhang, Z., Wang, J., & Zhao, H. (2021) · arXiv (Cornell University) · Cited by 2 · see citing works
Recent Advances in Automated Question Answering In Biomedical DomainPreprint
Baksi, K. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05937 · Cited by 1 · see citing works
RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory
Liu, J., Kong, Z., Yang, C., Yang, F., Li, T., Dong, P., Nanjekye, J., Tang, H., Yuan, G., Niu, W., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.04903
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
Zhou, P., Leon, B., Ying, X., Zhang, C., Shao, Y., Ye, Q., Chong, D., Jin, Z., Xie, C., Cao, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.19314
- Authors
- Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., & Manning, C. D.
- Affiliations
- Mila - Quebec Artificial Intelligence Institute (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1809.09600
- OpenAlex
- W2952862139
- Cited by
- 97
- References
- 16
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 1 · 2022: 5 · 2021: 31 · 2020: 38 · 2019: 12 · 2018: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1514268)
- Nvidia
- Université de Montréal
- Defense Advanced Research Projects Agency
- Office of Naval Research (N000141812861)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- multi-hop question answering, HotpotQA, explainable question answering, complex reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review